Jul 31 12:15:12.652449 ip-10-0-141-165 systemd[1]: Starting Kubernetes Kubelet...
Jul 31 12:15:13.073124 ip-10-0-141-165 kubenswrapper[2532]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 12:15:13.073124 ip-10-0-141-165 kubenswrapper[2532]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 31 12:15:13.073124 ip-10-0-141-165 kubenswrapper[2532]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 12:15:13.073124 ip-10-0-141-165 kubenswrapper[2532]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 31 12:15:13.073124 ip-10-0-141-165 kubenswrapper[2532]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 12:15:13.076330 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.076202    2532 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 12:15:13.082924 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082906    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.082924 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082922    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.082924 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082927    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.082924 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082932    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082936    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082940    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082944    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082949    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082953    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082956    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082960    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082963    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082967    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082972    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082975    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082979    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082984    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082988    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082992    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082996    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.082999    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083004    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083007    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.083151 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083011    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083015    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083019    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083023    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083027    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083030    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083034    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083038    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083042    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083046    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083051    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083055    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083059    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083064    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083069    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083074    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083078    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083082    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083090    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083097    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.083781 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083104    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083109    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083113    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083118    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083122    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083127    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083131    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083135    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083139    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083143    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083147    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083150    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083154    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083159    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083163    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083181    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083186    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083190    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083195    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083199    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.084271 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083203    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083208    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083212    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083216    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083220    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083225    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083230    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083235    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083240    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083245    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083249    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083253    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083257    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083262    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083266    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083270    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083273    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083278    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083282    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.084771 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083290    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083295    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083300    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083304    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083309    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083313    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083317    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083321    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083326    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083330    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083334    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083338    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083342    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083346    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083350    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083355    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.085365 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.083359    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.086148 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086132    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.086148 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086147    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086152    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086158    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086162    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086181    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086186    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086190    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086195    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086199    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086203    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086207    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086212    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086216    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086220    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086223    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086227    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086231    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086235    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086239    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086243    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.086281 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086248    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086251    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086255    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086259    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086264    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086268    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086272    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086276    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086280    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086284    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086288    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086292    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086296    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086300    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086303    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086307    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086311    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086317    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086321    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086325    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.087119 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086329    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086333    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086337    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086341    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086345    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086348    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086352    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086357    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086361    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086364    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086368    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086372    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086377    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086383    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086390    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086396    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086402    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086406    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086412    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.087958 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086418    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086422    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086426    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086430    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086435    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086439    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086445    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086450    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086455    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086459    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086464    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086469    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086473    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086477    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086481    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086486    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086491    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086495    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086499    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.088785 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086504    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086508    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086512    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086515    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086520    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086525    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086528    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086532    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086537    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086541    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086545    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086549    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086553    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086557    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086561    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086566    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086570    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086574    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086578    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.086582    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.089421 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087139    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087147    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087153    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087158    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087162    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087182    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087187    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087191    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087195    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087199    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087203    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087208    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087212    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087217    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087221    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087225    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087229    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087233    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087237    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087242    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.089893 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087246    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087250    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087254    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087258    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087262    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087266    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087270    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087274    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087278    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087284    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087290    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087295    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087299    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087304    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087309    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087313    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087317    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087323    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087329    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.090464 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087334    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087339    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087345    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087351    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087358    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087363    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087367    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087374    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087378    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087383    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087387    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087392    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087396    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087400    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087404    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087408    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087412    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087416    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087420    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.091093 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087424    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087428    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087432    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087436    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087440    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087443    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087447    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087451    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087456    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087460    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087464    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087468    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087472    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087476    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087480    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087484    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087488    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087493    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087498    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.091756 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087502    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087506    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087510    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087514    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087518    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087522    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087526    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087530    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087536    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087541    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087545    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087549    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087553    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087556    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087560    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087564    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087569    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087573    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087577    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087581    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.092215 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087585    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.087589    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087683    2532 flags.go:64] FLAG: --address="0.0.0.0"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087694    2532 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087708    2532 flags.go:64] FLAG: --anonymous-auth="true"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087714    2532 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087725    2532 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087731    2532 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087737    2532 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087743    2532 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087748    2532 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087753    2532 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087758    2532 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087763    2532 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087769    2532 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087774    2532 flags.go:64] FLAG: --cgroup-root=""
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087778    2532 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087783    2532 flags.go:64] FLAG: --client-ca-file=""
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087788    2532 flags.go:64] FLAG: --cloud-provider="external"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087793    2532 flags.go:64] FLAG: --cluster-dns="[]"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087798    2532 flags.go:64] FLAG: --cluster-domain=""
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087803    2532 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087808    2532 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087813    2532 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 31 12:15:13.092793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087818    2532 flags.go:64] FLAG: --container-log-max-files="5"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087824    2532 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087828    2532 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087833    2532 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087838    2532 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087843    2532 flags.go:64] FLAG: --contention-profiling="false"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087847    2532 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087852    2532 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087857    2532 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087861    2532 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087867    2532 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087872    2532 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087877    2532 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087881    2532 flags.go:64] FLAG: --enable-load-reader="false"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087886    2532 flags.go:64] FLAG: --enable-server="true"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087891    2532 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087898    2532 flags.go:64] FLAG: --event-burst="100"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087903    2532 flags.go:64] FLAG: --event-qps="50"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087908    2532 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087912    2532 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087917    2532 flags.go:64] FLAG: --eviction-hard=""
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087923    2532 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087927    2532 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087932    2532 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087937    2532 flags.go:64] FLAG: --eviction-soft=""
Jul 31 12:15:13.093379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087941    2532 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087946    2532 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087951    2532 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087955    2532 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087959    2532 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087963    2532 flags.go:64] FLAG: --fail-swap-on="true"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087968    2532 flags.go:64] FLAG: --feature-gates=""
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087973    2532 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087978    2532 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087983    2532 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087987    2532 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087992    2532 flags.go:64] FLAG: --healthz-port="10248"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.087997    2532 flags.go:64] FLAG: --help="false"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088002    2532 flags.go:64] FLAG: --hostname-override="ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088007    2532 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088011    2532 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088016    2532 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088021    2532 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088026    2532 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088030    2532 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088035    2532 flags.go:64] FLAG: --image-service-endpoint=""
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088039    2532 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088044    2532 flags.go:64] FLAG: --kube-api-burst="100"
Jul 31 12:15:13.094008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088049    2532 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088057    2532 flags.go:64] FLAG: --kube-api-qps="50"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088062    2532 flags.go:64] FLAG: --kube-reserved=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088067    2532 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088071    2532 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088076    2532 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088081    2532 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088086    2532 flags.go:64] FLAG: --lock-file=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088090    2532 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088094    2532 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088099    2532 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088107    2532 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088112    2532 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088117    2532 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088121    2532 flags.go:64] FLAG: --logging-format="text"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088126    2532 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088131    2532 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088135    2532 flags.go:64] FLAG: --manifest-url=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088140    2532 flags.go:64] FLAG: --manifest-url-header=""
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088146    2532 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088150    2532 flags.go:64] FLAG: --max-open-files="1000000"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088157    2532 flags.go:64] FLAG: --max-pods="110"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088162    2532 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088183    2532 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088188    2532 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 31 12:15:13.094594 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088193    2532 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088197    2532 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088202    2532 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088207    2532 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088216    2532 flags.go:64] FLAG: --node-status-max-images="50"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088220    2532 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088225    2532 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088232    2532 flags.go:64] FLAG: --pod-cidr=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088240    2532 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088248    2532 flags.go:64] FLAG: --pod-manifest-path=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088252    2532 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088257    2532 flags.go:64] FLAG: --pods-per-core="0"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088261    2532 flags.go:64] FLAG: --port="10250"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088266    2532 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088271    2532 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035ce2dcec10f654f"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088276    2532 flags.go:64] FLAG: --qos-reserved=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088281    2532 flags.go:64] FLAG: --read-only-port="10255"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088286    2532 flags.go:64] FLAG: --register-node="true"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088291    2532 flags.go:64] FLAG: --register-with-taints=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088296    2532 flags.go:64] FLAG: --registry-burst="10"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088301    2532 flags.go:64] FLAG: --registry-qps="5"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088305    2532 flags.go:64] FLAG: --reserved-cpus=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088310    2532 flags.go:64] FLAG: --reserved-memory=""
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088316    2532 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088320    2532 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 31 12:15:13.095235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088326    2532 flags.go:64] FLAG: --rotate-certificates="false"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088330    2532 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088335    2532 flags.go:64] FLAG: --runonce="false"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088339    2532 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088344    2532 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088349    2532 flags.go:64] FLAG: --seccomp-default="false"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088354    2532 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088358    2532 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088363    2532 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088368    2532 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088372    2532 flags.go:64] FLAG: --storage-driver-password="root"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088377    2532 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088381    2532 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088386    2532 flags.go:64] FLAG: --storage-driver-user="root"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088390    2532 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088397    2532 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088404    2532 flags.go:64] FLAG: --system-cgroups=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088408    2532 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088416    2532 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088420    2532 flags.go:64] FLAG: --tls-cert-file=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088425    2532 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088431    2532 flags.go:64] FLAG: --tls-min-version=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088435    2532 flags.go:64] FLAG: --tls-private-key-file=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088440    2532 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088445    2532 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 31 12:15:13.095861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088449    2532 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088454    2532 flags.go:64] FLAG: --v="2"
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088459    2532 flags.go:64] FLAG: --version="false"
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088465    2532 flags.go:64] FLAG: --vmodule=""
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088471    2532 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.088476    2532 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088657    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088664    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088669    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088674    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088679    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088683    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088688    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088692    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088696    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088700    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088704    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088711    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088717    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088722    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088727    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.096472 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088732    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088738    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088744    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088756    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088761    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088766    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088770    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088775    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088779    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088784    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088790    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088794    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088799    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088804    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088808    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088813    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088818    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088823    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088827    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.096977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088831    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088836    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088840    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088844    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088848    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088852    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088857    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088861    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088865    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088869    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088873    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088877    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088881    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088885    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088891    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088897    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088902    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088907    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088911    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.097566 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088915    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088920    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088924    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088928    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088932    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088936    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088940    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088944    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088949    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088953    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088957    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088961    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088966    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088970    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088974    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088978    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088982    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088986    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088991    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088995    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.098017 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.088999    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089003    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089007    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089011    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089015    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089019    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089023    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089029    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089035    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089039    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089043    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089048    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089052    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089056    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089060    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089064    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089070    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089074    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089078    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089082    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.098511 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089086    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089090    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089096    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089101    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089106    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.089110    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.090208    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.096662    2532 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.096678    2532 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.096695    2532 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.096702    2532 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096758    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096763    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096766    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096770    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.098977 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096773    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096776    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096778    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096781    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096784    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096787    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096789    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096792    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096794    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096797    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096800    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096802    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096805    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096807    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096810    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096812    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096815    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096817    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096820    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096822    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.099351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096825    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096827    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096830    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096832    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096835    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096839    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096842    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096845    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096847    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096850    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096852    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096855    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096857    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096860    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096863    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096865    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096868    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096870    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096873    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.099818 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096875    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096878    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096880    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096883    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096886    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096888    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096892    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096896    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096900    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096904    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096907    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096909    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096912    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096915    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096917    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096920    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096923    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096925    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096929    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.100279 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096932    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096935    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096938    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096940    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096942    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096945    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096948    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096950    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096953    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096955    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096958    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096960    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096962    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096965    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096968    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096970    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096973    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096975    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096978    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096981    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.100731 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096983    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096986    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096988    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096991    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096994    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096996    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.096999    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097002    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097004    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097008    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097011    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097014    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097016    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097020    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097023    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097026    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.101265 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097029    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.097034    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097156    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097161    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097164    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097181    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097185    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097188    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097191    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097194    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097197    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097199    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097201    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097204    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097206    2532 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 12:15:13.101656 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097209    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097211    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097214    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097216    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097219    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097221    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097224    2532 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097226    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097229    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097231    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097234    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097236    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097239    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097241    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097244    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097247    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097250    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097253    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097255    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097257    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 12:15:13.102022 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097260    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097263    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097267    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097269    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097272    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097274    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097277    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097279    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097282    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097284    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097286    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097289    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097293    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097296    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097299    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097302    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097304    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097307    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097310    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 12:15:13.102518 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097313    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097315    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097318    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097320    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097323    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097325    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097328    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097330    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097333    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097336    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097339    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097341    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097344    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097346    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097349    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097351    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097353    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097356    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097358    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097360    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 12:15:13.102955 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097364    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097367    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097369    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097372    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097375    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097378    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097380    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097383    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097386    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097389    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097392    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097394    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097402    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097406    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097409    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097411    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097414    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097417    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097419    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097422    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 12:15:13.103436 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097425    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097427    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097430    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097433    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097436    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097438    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:13.097440    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.097445    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.097548    2532 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.099648    2532 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.101071    2532 server.go:1019] "Starting client certificate rotation"
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.101166    2532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 12:15:13.103895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.101227    2532 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 12:15:13.104192 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.103996    2532 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 12:15:13.106695 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.106672    2532 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 12:15:13.117007 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.116985    2532 log.go:25] "Validated CRI v1 runtime API"
Jul 31 12:15:13.118951 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.118938    2532 log.go:25] "Validated CRI v1 image API"
Jul 31 12:15:13.119705 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.119684    2532 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 31 12:15:13.122372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.122349    2532 fs.go:135] Filesystem UUIDs: map[1629b0ff-c56b-4d4f-964e-09484e593ec3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8dc9e8a2-47cd-4ab3-974b-b4526706a96f:/dev/nvme0n1p3]
Jul 31 12:15:13.122423 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.122372    2532 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 31 12:15:13.127102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.126998    2532 manager.go:216] Machine: {Timestamp:2026-07-31 12:15:13.125846261 +0000 UTC m=+0.357711003 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196524 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cdc7cd1612149e3ec9779efdbd4ec SystemUUID:ec2cdc7c-d161-2149-e3ec-9779efdbd4ec BootID:0007c9f3-090f-4c72-aee9-a0f47b158e53 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:90:8b:64:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:90:8b:64:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:a7:81:b9:ff:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 31 12:15:13.127102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.127091    2532 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 31 12:15:13.127242 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.127158    2532 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 31 12:15:13.128543 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.128530    2532 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 12:15:13.129957 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.129937    2532 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 12:15:13.130101 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.129961    2532 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 31 12:15:13.131115 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.131105    2532 topology_manager.go:138] "Creating topology manager with none policy"
Jul 31 12:15:13.131154 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.131117    2532 container_manager_linux.go:309] "Creating device plugin manager"
Jul 31 12:15:13.131154 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.131125    2532 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 12:15:13.131711 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.131701    2532 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 12:15:13.132502 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.132493    2532 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 31 12:15:13.136078 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136067    2532 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 12:15:13.136192 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136183    2532 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 31 12:15:13.136233 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136210    2532 kubelet.go:486] "Attempting to sync node with API server"
Jul 31 12:15:13.136233 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136220    2532 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 31 12:15:13.136233 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136232    2532 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 31 12:15:13.136349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136242    2532 kubelet.go:398] "Adding apiserver pod source"
Jul 31 12:15:13.136349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136250    2532 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 12:15:13.136555 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136543    2532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 12:15:13.136589 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.136561    2532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 12:15:13.145787 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.145771    2532 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 31 12:15:13.147156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.147144    2532 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 31 12:15:13.147215 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.147190    2532 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 31 12:15:13.148337 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148326    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 31 12:15:13.148368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148342    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 31 12:15:13.148368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148347    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 31 12:15:13.148368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148352    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 31 12:15:13.148368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148360    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 31 12:15:13.148368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148365    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148373    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148378    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148383    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148388    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148397    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 31 12:15:13.148514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.148402    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 31 12:15:13.149115 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.149105    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 31 12:15:13.149142 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.149115    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 31 12:15:13.151092 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.151061    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 31 12:15:13.151201 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.151157    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 31 12:15:13.151304 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.151280    2532 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 31 12:15:13.152487 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.152471    2532 server.go:1268] "Started kubelet"
Jul 31 12:15:13.152728 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.152623    2532 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 31 12:15:13.152728 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.152677    2532 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 31 12:15:13.152852 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.152755    2532 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 31 12:15:13.153148 ip-10-0-141-165 systemd[1]: Started Kubernetes Kubelet.
Jul 31 12:15:13.153982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.153966    2532 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 31 12:15:13.154366 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.154355    2532 server.go:310] "Adding debug handlers to kubelet server"
Jul 31 12:15:13.158367 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.158344    2532 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 31 12:15:13.159219 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159201    2532 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 12:15:13.159219 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159204    2532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 31 12:15:13.159399 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.159380    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.159793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159775    2532 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 31 12:15:13.159865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159796    2532 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 31 12:15:13.159865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159774    2532 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 31 12:15:13.159961 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159942    2532 reconstruct.go:97] "Volume reconstruction finished"
Jul 31 12:15:13.159961 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.159949    2532 reconciler.go:29] "Reconciler: start to sync state"
Jul 31 12:15:13.160395 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160370    2532 factory.go:153] Registering CRI-O factory
Jul 31 12:15:13.160477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160410    2532 factory.go:223] Registration of the crio container factory successfully
Jul 31 12:15:13.160477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160460    2532 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 31 12:15:13.160477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160469    2532 factory.go:55] Registering systemd factory
Jul 31 12:15:13.160477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160478    2532 factory.go:223] Registration of the systemd container factory successfully
Jul 31 12:15:13.160631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160497    2532 factory.go:103] Registering Raw factory
Jul 31 12:15:13.160631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160520    2532 manager.go:1196] Started watching for new ooms in manager
Jul 31 12:15:13.160958 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.160944    2532 manager.go:349] Starting recovery of all containers
Jul 31 12:15:13.167454 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.167428    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 31 12:15:13.167574 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.167547    2532 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 31 12:15:13.168582 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.167479    2532 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-165.ec2.internal.18c75ee496dee9aa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-165.ec2.internal,UID:ip-10-0-141-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-165.ec2.internal,},FirstTimestamp:2026-07-31 12:15:13.15243665 +0000 UTC m=+0.384301392,LastTimestamp:2026-07-31 12:15:13.15243665 +0000 UTC m=+0.384301392,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-165.ec2.internal,}"
Jul 31 12:15:13.170428 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.170344    2532 manager.go:354] Recovery completed
Jul 31 12:15:13.175948 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.175934    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.177811 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.177797    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.177871 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.177822    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.177871 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.177831    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.178283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178263    2532 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 31 12:15:13.178283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178275    2532 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 31 12:15:13.178385 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178292    2532 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 12:15:13.178385 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178325    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.178966 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178954    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.179011 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178977    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.179011 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.178986    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.179405 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.179395    2532 policy_none.go:49] "None policy: Start"
Jul 31 12:15:13.179436 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.179410    2532 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 31 12:15:13.179436 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.179419    2532 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 31 12:15:13.179840 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.179776    2532 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-165.ec2.internal.18c75ee498621566  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-165.ec2.internal,UID:ip-10-0-141-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-165.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-165.ec2.internal,},FirstTimestamp:2026-07-31 12:15:13.177810278 +0000 UTC m=+0.409675019,LastTimestamp:2026-07-31 12:15:13.177810278 +0000 UTC m=+0.409675019,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-165.ec2.internal,}"
Jul 31 12:15:13.180559 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.180548    2532 policy_none.go:47] "Start"
Jul 31 12:15:13.188064 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.187925    2532 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-165.ec2.internal.18c75ee498625421  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-165.ec2.internal,UID:ip-10-0-141-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-165.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-165.ec2.internal,},FirstTimestamp:2026-07-31 12:15:13.177826337 +0000 UTC m=+0.409691079,LastTimestamp:2026-07-31 12:15:13.177826337 +0000 UTC m=+0.409691079,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-165.ec2.internal,}"
Jul 31 12:15:13.197696 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.197680    2532 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hqxw7"
Jul 31 12:15:13.200218 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.200144    2532 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-165.ec2.internal.18c75ee498627597  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-165.ec2.internal,UID:ip-10-0-141-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-165.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-165.ec2.internal,},FirstTimestamp:2026-07-31 12:15:13.177834903 +0000 UTC m=+0.409699644,LastTimestamp:2026-07-31 12:15:13.177834903 +0000 UTC m=+0.409699644,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-165.ec2.internal,}"
Jul 31 12:15:13.205342 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.205323    2532 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hqxw7"
Jul 31 12:15:13.209716 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.209704    2532 manager.go:337] "Starting Device Plugin manager"
Jul 31 12:15:13.209782 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.209729    2532 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 31 12:15:13.209782 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.209738    2532 server.go:87] "Starting device plugin registration server"
Jul 31 12:15:13.209921 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.209910    2532 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 31 12:15:13.209974 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.209925    2532 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 31 12:15:13.210025 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.209982    2532 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 31 12:15:13.210067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.210046    2532 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 31 12:15:13.210067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.210054    2532 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 31 12:15:13.210583 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.210566    2532 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 31 12:15:13.210653 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.210615    2532 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.310556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.310536    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.311660 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.311343    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.311660 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.311368    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.311660 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.311379    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.311660 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.311399    2532 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.319786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.319773    2532 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.319827 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.319792    2532 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-165.ec2.internal\": node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.335824 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.335783    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.338408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.338384    2532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 31 12:15:13.339401 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.339384    2532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 31 12:15:13.339464 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.339415    2532 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 31 12:15:13.339464 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.339433    2532 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 31 12:15:13.339464 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.339460    2532 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 31 12:15:13.341433 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.341418    2532 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 12:15:13.436711 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.436686    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.439845 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.439830    2532 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"]
Jul 31 12:15:13.439899 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.439889    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.440623 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.440600    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.440715 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.440632    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.440715 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.440645    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.442388 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.442375    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.442533 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.442512    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.442576 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.442542    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.443179 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443162    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.443179 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443155    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.443305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443201    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.443305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443202    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.443305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443225    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.443305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.443212    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.444612 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.444594    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.444695 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.444624    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:13.445502 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.445475    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:13.445616 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.445517    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:13.445616 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.445533    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:13.461059 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.461033    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.461148 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.461067    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.461148 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.461093    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7711a431e63758ca060bb86d740f9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"e7711a431e63758ca060bb86d740f9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.463812 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.463796    2532 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-165.ec2.internal\" not found" node="ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.467793 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.467779    2532 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-165.ec2.internal\" not found" node="ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.537429 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.537412    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.562021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562001    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.562082 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562026    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.562082 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562044    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7711a431e63758ca060bb86d740f9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"e7711a431e63758ca060bb86d740f9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.562082 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562079    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7711a431e63758ca060bb86d740f9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"e7711a431e63758ca060bb86d740f9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.562229 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562116    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.562229 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.562116    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2321338cbfd1a0e5e9cde68af13d0e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"c2321338cbfd1a0e5e9cde68af13d0e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.639085 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.639039    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.745049 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.745013    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.765952 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.765928    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.771138 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:13.770668    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:13.846835 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.846802    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:13.947828 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:13.947753    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.054006 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.053981    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.102091 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.102057    2532 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 31 12:15:14.102535 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.102228    2532 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 12:15:14.154973 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.154945    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.209490 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.209450    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 12:10:13 +0000 UTC" deadline="2028-02-12 22:10:24.581708021 +0000 UTC"
Jul 31 12:15:14.209490 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.209486    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13473h55m10.372227145s"
Jul 31 12:15:14.239292 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.239272    2532 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 12:15:14.255403 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.255358    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.269095 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.269060    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 12:15:14.270466 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.270443    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 12:15:14.270597 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.270482    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 12:15:14.270597 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.270499    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 31 12:15:14.275932 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.275914    2532 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 12:15:14.326229 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:14.326189    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7711a431e63758ca060bb86d740f9a6.slice/crio-e90bbcb54452b641321e2ebdc5799b9e8cea491f890ff87791bde2c04dfa571c WatchSource:0}: Error finding container e90bbcb54452b641321e2ebdc5799b9e8cea491f890ff87791bde2c04dfa571c: Status 404 returned error can't find the container with id e90bbcb54452b641321e2ebdc5799b9e8cea491f890ff87791bde2c04dfa571c
Jul 31 12:15:14.330409 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.330394    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 12:15:14.342417 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.342370    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" event={"ID":"e7711a431e63758ca060bb86d740f9a6","Type":"ContainerStarted","Data":"e90bbcb54452b641321e2ebdc5799b9e8cea491f890ff87791bde2c04dfa571c"}
Jul 31 12:15:14.355521 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.355498    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.384025 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:14.384001    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2321338cbfd1a0e5e9cde68af13d0e1.slice/crio-4d5ff2a3a72ed3b9cedf92d2e7c2f283e1eeb492294e6ba6977a1cb5b132c56d WatchSource:0}: Error finding container 4d5ff2a3a72ed3b9cedf92d2e7c2f283e1eeb492294e6ba6977a1cb5b132c56d: Status 404 returned error can't find the container with id 4d5ff2a3a72ed3b9cedf92d2e7c2f283e1eeb492294e6ba6977a1cb5b132c56d
Jul 31 12:15:14.460450 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.460365    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.561566 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.561532    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.662454 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:14.662415    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 31 12:15:14.678142 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.678123    2532 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 12:15:14.760496 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.760417    2532 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:14.772294 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.772262    2532 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 12:15:14.773364 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.773335    2532 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 31 12:15:14.782395 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:14.782371    2532 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 12:15:15.138388 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.138310    2532 apiserver.go:52] "Watching apiserver"
Jul 31 12:15:15.147566 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.147538    2532 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 31 12:15:15.148125 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.148099    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-59h74","kube-system/konnectivity-agent-j6kr9","openshift-image-registry/node-ca-dgnbk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal","openshift-multus/multus-zwgfj","openshift-network-diagnostics/network-check-target-jmffq","openshift-network-operator/iptables-alerter-pz4m5","openshift-ovn-kubernetes/ovnkube-node-7lw6b","kube-system/global-pull-secret-syncer-xqkmw","kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk","openshift-cluster-node-tuning-operator/tuned-2x9rc","openshift-dns/node-resolver-9lkp6","openshift-multus/multus-additional-cni-plugins-t775z"]
Jul 31 12:15:15.150066 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.150043    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.151457 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.151427    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.152555 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.152530    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.153204 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.153176    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.153478 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.153461    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tl586\""
Jul 31 12:15:15.153566 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.153507    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.154156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.154036    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.154156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.154060    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 31 12:15:15.154346 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.154156    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:15.154346 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.154305    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 31 12:15:15.154638 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.154616    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 31 12:15:15.155066 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.155046    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.155152 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.155139    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kmrbj\""
Jul 31 12:15:15.155362 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.155343    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9zhn4\""
Jul 31 12:15:15.155611 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.155593    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.155685 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.155602    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.157137 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.157029    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:15.157137 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.157103    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:15.157754 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.157735    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 31 12:15:15.157949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.157925    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.158386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.158353    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.158386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.158375    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nktfx\""
Jul 31 12:15:15.158658 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.158638    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 31 12:15:15.158766 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.158686    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.160310 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.160291    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.160403 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.160363    2532 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 31 12:15:15.161263 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.161231    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkktr\""
Jul 31 12:15:15.161356 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.161269    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.161356 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.161288    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 31 12:15:15.161555 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.161534    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.162113 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.162095    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.162221 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.162151    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:15.162580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.162560    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 31 12:15:15.162855 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.162835    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 31 12:15:15.163122 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.163106    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.163199 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.163165    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jn95k\""
Jul 31 12:15:15.165325 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.164598    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.166271 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.165869    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 31 12:15:15.166271 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.166065    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 31 12:15:15.167272 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.166627    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.167413 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.167377    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.169634 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.169601    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 31 12:15:15.169853 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.169836    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 31 12:15:15.170074 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.170027    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5bg96\""
Jul 31 12:15:15.170372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.170353    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 31 12:15:15.171067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.170708    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.171067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.170743    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.171067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.170810    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fscsb\""
Jul 31 12:15:15.172314 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172286    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-netns\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172393 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172319    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-env-overrides\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172393 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172344    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-kubelet\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.172393 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172368    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-systemd-units\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172418    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cni-binary-copy\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172447    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-bin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172472    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-conf-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172496    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a66df52f-4829-48d2-b940-6eba62a720b9-konnectivity-ca\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172524    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172551    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172592    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6r49\" (UniqueName: \"kubernetes.io/projected/2d8617f0-21ff-447c-a82c-176a4d647a9b-kube-api-access-p6r49\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172633    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m6kt\" (UniqueName: \"kubernetes.io/projected/50430d0c-78a9-4876-8dfe-9753de79bb9a-kube-api-access-2m6kt\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172708    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgpfc\" (UniqueName: \"kubernetes.io/projected/aa412f99-c8df-40db-9f64-a3e4122b5834-kube-api-access-dgpfc\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172745    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hlqz\" (UniqueName: \"kubernetes.io/projected/f581f940-84e7-41b3-a723-29e3e33fceac-kube-api-access-4hlqz\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172774    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-config\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172798    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-multus-certs\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.172843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172824    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-k8s-cni-cncf-io\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172864    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172866    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-netns\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172953    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50430d0c-78a9-4876-8dfe-9753de79bb9a-serviceca\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.172980    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa412f99-c8df-40db-9f64-a3e4122b5834-host-slash\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173006    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173030    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-var-lib-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173060    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-kubelet-config\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.173187 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173085    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5jk\" (UniqueName: \"kubernetes.io/projected/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-kube-api-access-lv5jk\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173185    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-system-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173226    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cnibin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173256    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-daemon-config\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173272    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hvdg\" (UniqueName: \"kubernetes.io/projected/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-kube-api-access-6hvdg\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173322    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-tmp-dir\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173359    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173377    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-slash\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173394    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50430d0c-78a9-4876-8dfe-9753de79bb9a-host\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173413    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-hostroot\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173430    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-hosts-file\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173447    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-netd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173462    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-script-lib\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173477    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173494    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-socket-dir-parent\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173510    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-bin\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173528    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa412f99-c8df-40db-9f64-a3e4122b5834-iptables-alerter-script\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.173604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173578    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173595    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-os-release\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173612    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-systemd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173632    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-log-socket\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173652    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173671    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-multus\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173703    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-kubelet\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173721    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173736    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-dbus\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173757    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a66df52f-4829-48d2-b940-6eba62a720b9-agent-certs\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173780    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-etc-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173804    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-node-log\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173843    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-ovn\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.174445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.173867    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-etc-kubernetes\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.175344 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.175325    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dgj52\""
Jul 31 12:15:15.175426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.175372    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:15:15.175426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.175414    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 31 12:15:15.175706 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.175688    2532 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 12:15:15.197469 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.197449    2532 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vv8cl"
Jul 31 12:15:15.203357 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.203339    2532 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vv8cl"
Jul 31 12:15:15.261311 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.261290    2532 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 31 12:15:15.274324 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274256    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-tmp-dir\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.274324 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274296    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.274324 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274325    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-slash\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274349    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50430d0c-78a9-4876-8dfe-9753de79bb9a-host\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274374    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-hostroot\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274405    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-modprobe-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274413    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-slash\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274432    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-hosts-file\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.274441    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274473    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-netd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274498    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-hosts-file\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274501    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-script-lib\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.274556 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.274553    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:15.774494595 +0000 UTC m=+3.006359343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274595    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50430d0c-78a9-4876-8dfe-9753de79bb9a-host\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274657    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-hostroot\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274682    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-tmp-dir\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274756    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274786    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-socket-dir-parent\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274788    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-netd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274813    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-cnibin\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274849    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-bin\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274875    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa412f99-c8df-40db-9f64-a3e4122b5834-iptables-alerter-script\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.274896    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274916    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.274950    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:15.774930326 +0000 UTC m=+3.006795072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274967    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-os-release\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.274995    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-systemd\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275048    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-systemd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275024    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-socket-dir-parent\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275076    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-log-socket\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275078    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275079    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-os-release\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275122    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-systemd\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275108    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-cni-bin\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275141    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275122    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-log-socket\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275259    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-multus\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275173    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275294    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-kubelet\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275299    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-multus\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275322    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275348    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-dbus\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275356    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-kubelet\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275373    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a66df52f-4829-48d2-b940-6eba62a720b9-agent-certs\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275385    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-script-lib\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275413    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-dbus\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275517    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-etc-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.275914 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275552    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-socket-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275562    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa412f99-c8df-40db-9f64-a3e4122b5834-iptables-alerter-script\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275579    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mhqc\" (UniqueName: \"kubernetes.io/projected/49bfc914-42f7-4768-9448-20c7ce0121ce-kube-api-access-2mhqc\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275617    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-etc-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275609    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-node-log\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275704    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-ovn\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275730    2532 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275760    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-node-log\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275760    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-etc-kubernetes\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275805    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-ovn\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275861    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysconfig\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275897    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-etc-kubernetes\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.275948    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276008    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-netns\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-env-overrides\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276072    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-kubelet\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276112    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-run-netns\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276134    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-kubelet\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.276873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276206    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-registration-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276253    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-sys-fs\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276297    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxqst\" (UniqueName: \"kubernetes.io/projected/46034009-6097-40de-bd1b-25569aa25011-kube-api-access-bxqst\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276344    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-lib-modules\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276384    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-systemd-units\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276426    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-systemd-units\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276457    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cni-binary-copy\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276483    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-env-overrides\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276507    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-bin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276544    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-conf-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276581    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-var-lib-cni-bin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276571    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276613    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-conf-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276638    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a66df52f-4829-48d2-b940-6eba62a720b9-konnectivity-ca\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276664    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276688    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276734    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6r49\" (UniqueName: \"kubernetes.io/projected/2d8617f0-21ff-447c-a82c-176a4d647a9b-kube-api-access-p6r49\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.277721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276756    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m6kt\" (UniqueName: \"kubernetes.io/projected/50430d0c-78a9-4876-8dfe-9753de79bb9a-kube-api-access-2m6kt\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276777    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276797    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgpfc\" (UniqueName: \"kubernetes.io/projected/aa412f99-c8df-40db-9f64-a3e4122b5834-kube-api-access-dgpfc\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276818    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hlqz\" (UniqueName: \"kubernetes.io/projected/f581f940-84e7-41b3-a723-29e3e33fceac-kube-api-access-4hlqz\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276842    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-config\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276875    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-multus-certs\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276912    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276951    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-kubernetes\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.276982    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-conf\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277004    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-k8s-cni-cncf-io\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277025    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-netns\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277063    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmm8z\" (UniqueName: \"kubernetes.io/projected/fe0053fa-6869-42c1-8011-52836ccf5f5f-kube-api-access-xmm8z\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277083    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277124    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-var-lib-kubelet\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277149    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277266    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50430d0c-78a9-4876-8dfe-9753de79bb9a-serviceca\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277292    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-run\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.278419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277302    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a66df52f-4829-48d2-b940-6eba62a720b9-konnectivity-ca\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277353    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-run-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277014    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cni-binary-copy\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277546    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277604    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-sys\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277633    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-host\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277675    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277715    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa412f99-c8df-40db-9f64-a3e4122b5834-host-slash\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277760    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-k8s-cni-cncf-io\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277764    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa412f99-c8df-40db-9f64-a3e4122b5834-host-slash\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277803    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-netns\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277818    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovnkube-config\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277837    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-host-run-multus-certs\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277882    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277926    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-var-lib-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.277958    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d8617f0-21ff-447c-a82c-176a4d647a9b-var-lib-openvswitch\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278007    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-kubelet-config\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.279258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278035    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-tuned\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278074    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/50430d0c-78a9-4876-8dfe-9753de79bb9a-serviceca\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278092    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01539827-8873-463d-89b1-ff47d4e4b2a8-kubelet-config\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278109    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-tmp\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278158    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-os-release\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278208    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5jk\" (UniqueName: \"kubernetes.io/projected/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-kube-api-access-lv5jk\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278237    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-system-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278290    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cnibin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278314    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-daemon-config\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278331    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-system-cni-dir\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278346    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hvdg\" (UniqueName: \"kubernetes.io/projected/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-kube-api-access-6hvdg\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278373    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-device-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278389    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-cnibin\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278399    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-system-cni-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.278885    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-multus-daemon-config\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.279987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.279713    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a66df52f-4829-48d2-b940-6eba62a720b9-agent-certs\") pod \"konnectivity-agent-j6kr9\" (UID: \"a66df52f-4829-48d2-b940-6eba62a720b9\") " pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.280831 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.280105    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d8617f0-21ff-447c-a82c-176a4d647a9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.285689 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.285381    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:15.285689 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.285407    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:15.285689 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.285420    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:15.285689 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.285470    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:15.785455864 +0000 UTC m=+3.017320629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:15.287432 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.287406    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgpfc\" (UniqueName: \"kubernetes.io/projected/aa412f99-c8df-40db-9f64-a3e4122b5834-kube-api-access-dgpfc\") pod \"iptables-alerter-pz4m5\" (UID: \"aa412f99-c8df-40db-9f64-a3e4122b5834\") " pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.288166 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.288144    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m6kt\" (UniqueName: \"kubernetes.io/projected/50430d0c-78a9-4876-8dfe-9753de79bb9a-kube-api-access-2m6kt\") pod \"node-ca-dgnbk\" (UID: \"50430d0c-78a9-4876-8dfe-9753de79bb9a\") " pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.288656 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.288634    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5jk\" (UniqueName: \"kubernetes.io/projected/81c723fb-b45f-49ee-9a71-0f00ecb73e6b-kube-api-access-lv5jk\") pod \"node-resolver-9lkp6\" (UID: \"81c723fb-b45f-49ee-9a71-0f00ecb73e6b\") " pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.288656 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.288643    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hvdg\" (UniqueName: \"kubernetes.io/projected/fd1c837c-3d3d-43f7-a61d-8a8774cd91cb-kube-api-access-6hvdg\") pod \"multus-zwgfj\" (UID: \"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb\") " pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.289369 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.289343    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hlqz\" (UniqueName: \"kubernetes.io/projected/f581f940-84e7-41b3-a723-29e3e33fceac-kube-api-access-4hlqz\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.290597 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.290576    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6r49\" (UniqueName: \"kubernetes.io/projected/2d8617f0-21ff-447c-a82c-176a4d647a9b-kube-api-access-p6r49\") pod \"ovnkube-node-7lw6b\" (UID: \"2d8617f0-21ff-447c-a82c-176a4d647a9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.345097 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.345058    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"c2321338cbfd1a0e5e9cde68af13d0e1","Type":"ContainerStarted","Data":"4d5ff2a3a72ed3b9cedf92d2e7c2f283e1eeb492294e6ba6977a1cb5b132c56d"}
Jul 31 12:15:15.379028 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.378998    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-systemd\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379139 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379046    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-socket-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379139 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379075    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mhqc\" (UniqueName: \"kubernetes.io/projected/49bfc914-42f7-4768-9448-20c7ce0121ce-kube-api-access-2mhqc\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.379139 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379103    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysconfig\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379139 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379127    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.379139 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379130    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-systemd\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379153    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-registration-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379207    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-sys-fs\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379225    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysconfig\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379241    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxqst\" (UniqueName: \"kubernetes.io/projected/46034009-6097-40de-bd1b-25569aa25011-kube-api-access-bxqst\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379237    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-socket-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379268    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-lib-modules\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379299    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379329    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379358    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379382    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-kubernetes\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379404    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.379429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379407    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-conf\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379460    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmm8z\" (UniqueName: \"kubernetes.io/projected/fe0053fa-6869-42c1-8011-52836ccf5f5f-kube-api-access-xmm8z\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379501    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379527    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-var-lib-kubelet\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379557    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-conf\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379557    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379605    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-run\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379621    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-sysctl-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379634    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-sys\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379660    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-host\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379686    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379719    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-lib-modules\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379732    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-tuned\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379758    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-tmp\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379783    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-os-release\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379814    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-device-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379840    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-system-cni-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379883    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-modprobe-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.379926    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-cnibin\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380004    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-cnibin\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380091    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380142    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-kubernetes\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380194    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380193    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-sys-fs\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380279    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-run\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380287    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380341    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-host\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380342    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-sys\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380397    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-device-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380398    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-os-release\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380621    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-registration-dir\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380823    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49bfc914-42f7-4768-9448-20c7ce0121ce-system-cni-dir\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380865    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46034009-6097-40de-bd1b-25569aa25011-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380871    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-var-lib-kubelet\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.380918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380915    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49bfc914-42f7-4768-9448-20c7ce0121ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.381734 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.380968    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-modprobe-d\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.383265 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.383230    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-etc-tuned\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.383372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.383273    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0053fa-6869-42c1-8011-52836ccf5f5f-tmp\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.387496 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.387462    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxqst\" (UniqueName: \"kubernetes.io/projected/46034009-6097-40de-bd1b-25569aa25011-kube-api-access-bxqst\") pod \"aws-ebs-csi-driver-node-sqtgk\" (UID: \"46034009-6097-40de-bd1b-25569aa25011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.388379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.388361    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmm8z\" (UniqueName: \"kubernetes.io/projected/fe0053fa-6869-42c1-8011-52836ccf5f5f-kube-api-access-xmm8z\") pod \"tuned-2x9rc\" (UID: \"fe0053fa-6869-42c1-8011-52836ccf5f5f\") " pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.388495 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.388383    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mhqc\" (UniqueName: \"kubernetes.io/projected/49bfc914-42f7-4768-9448-20c7ce0121ce-kube-api-access-2mhqc\") pod \"multus-additional-cni-plugins-t775z\" (UID: \"49bfc914-42f7-4768-9448-20c7ce0121ce\") " pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.461120 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.461095    2532 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 12:15:15.467252 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.467230    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lkp6"
Jul 31 12:15:15.476293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.476272    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zwgfj"
Jul 31 12:15:15.485565 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.485542    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:15.491512 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.491492    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dgnbk"
Jul 31 12:15:15.499077 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.499053    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pz4m5"
Jul 31 12:15:15.506844 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.506825    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:15.514532 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.514515    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t775z"
Jul 31 12:15:15.529257 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.529238    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk"
Jul 31 12:15:15.534014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.533993    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2x9rc"
Jul 31 12:15:15.782598 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.782565    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:15.782803 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.782611    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:15.782803 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.782747    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:15.782803 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.782744    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:15.782954 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.782809    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:16.78278922 +0000 UTC m=+4.014653960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:15.782954 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.782866    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:16.782845682 +0000 UTC m=+4.014710435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:15.883892 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:15.883857    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:15.884069 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.884027    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:15.884069 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.884049    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:15.884069 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.884058    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:15.884260 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:15.884114    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:16.884096444 +0000 UTC m=+4.115961198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:16.206216 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.206139    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 12:10:15 +0000 UTC" deadline="2028-03-27 14:03:14.319148733 +0000 UTC"
Jul 31 12:15:16.206216 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.206169    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14521h47m58.112983117s"
Jul 31 12:15:16.261003 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.260971    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50430d0c_78a9_4876_8dfe_9753de79bb9a.slice/crio-a197a27a017fb96e19b4d489a460c79137a8ed47614364f72d5ae18eadfe0ccd WatchSource:0}: Error finding container a197a27a017fb96e19b4d489a460c79137a8ed47614364f72d5ae18eadfe0ccd: Status 404 returned error can't find the container with id a197a27a017fb96e19b4d489a460c79137a8ed47614364f72d5ae18eadfe0ccd
Jul 31 12:15:16.264254 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.264094    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66df52f_4829_48d2_b940_6eba62a720b9.slice/crio-a51b899155dc7deef1589e17c90cc623776f0238956a1f6789223add5ad3b32e WatchSource:0}: Error finding container a51b899155dc7deef1589e17c90cc623776f0238956a1f6789223add5ad3b32e: Status 404 returned error can't find the container with id a51b899155dc7deef1589e17c90cc623776f0238956a1f6789223add5ad3b32e
Jul 31 12:15:16.266482 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.266451    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1c837c_3d3d_43f7_a61d_8a8774cd91cb.slice/crio-2612e0acaf907da6cd866303f4f94e8c798c5cec957febb72996bf2eec9e900a WatchSource:0}: Error finding container 2612e0acaf907da6cd866303f4f94e8c798c5cec957febb72996bf2eec9e900a: Status 404 returned error can't find the container with id 2612e0acaf907da6cd866303f4f94e8c798c5cec957febb72996bf2eec9e900a
Jul 31 12:15:16.267367 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.267282    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0053fa_6869_42c1_8011_52836ccf5f5f.slice/crio-0328d15f0bf27fdd155c747d788e647b6e0ecf5037fee2caa894de9b9dd82d6b WatchSource:0}: Error finding container 0328d15f0bf27fdd155c747d788e647b6e0ecf5037fee2caa894de9b9dd82d6b: Status 404 returned error can't find the container with id 0328d15f0bf27fdd155c747d788e647b6e0ecf5037fee2caa894de9b9dd82d6b
Jul 31 12:15:16.269010 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.268886    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c723fb_b45f_49ee_9a71_0f00ecb73e6b.slice/crio-ea37241cce700239d51e9d0392a5da2ae5db938c9ee32fa07a493b98f4b9b8be WatchSource:0}: Error finding container ea37241cce700239d51e9d0392a5da2ae5db938c9ee32fa07a493b98f4b9b8be: Status 404 returned error can't find the container with id ea37241cce700239d51e9d0392a5da2ae5db938c9ee32fa07a493b98f4b9b8be
Jul 31 12:15:16.269447 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.269425    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa412f99_c8df_40db_9f64_a3e4122b5834.slice/crio-409eee29268233d73bd0983f28f0e1f5c12b6a01984e972f95bc42667fa2f5f8 WatchSource:0}: Error finding container 409eee29268233d73bd0983f28f0e1f5c12b6a01984e972f95bc42667fa2f5f8: Status 404 returned error can't find the container with id 409eee29268233d73bd0983f28f0e1f5c12b6a01984e972f95bc42667fa2f5f8
Jul 31 12:15:16.271004 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:16.270980    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46034009_6097_40de_bd1b_25569aa25011.slice/crio-4c29ab21f187455ba0f07d33109997327cb1905e568cb0afb28ff45840874408 WatchSource:0}: Error finding container 4c29ab21f187455ba0f07d33109997327cb1905e568cb0afb28ff45840874408: Status 404 returned error can't find the container with id 4c29ab21f187455ba0f07d33109997327cb1905e568cb0afb28ff45840874408
Jul 31 12:15:16.340227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.340055    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:16.340377 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.340355    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:16.348308 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.348269    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2x9rc" event={"ID":"fe0053fa-6869-42c1-8011-52836ccf5f5f","Type":"ContainerStarted","Data":"0328d15f0bf27fdd155c747d788e647b6e0ecf5037fee2caa894de9b9dd82d6b"}
Jul 31 12:15:16.349680 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.349656    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zwgfj" event={"ID":"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb","Type":"ContainerStarted","Data":"2612e0acaf907da6cd866303f4f94e8c798c5cec957febb72996bf2eec9e900a"}
Jul 31 12:15:16.351259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.351235    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"a2f635922d8d09c149a0012a7e3b9b693dfe5e79fca28c17575f043102a7bbcc"}
Jul 31 12:15:16.352573 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.352540    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j6kr9" event={"ID":"a66df52f-4829-48d2-b940-6eba62a720b9","Type":"ContainerStarted","Data":"a51b899155dc7deef1589e17c90cc623776f0238956a1f6789223add5ad3b32e"}
Jul 31 12:15:16.353779 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.353758    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dgnbk" event={"ID":"50430d0c-78a9-4876-8dfe-9753de79bb9a","Type":"ContainerStarted","Data":"a197a27a017fb96e19b4d489a460c79137a8ed47614364f72d5ae18eadfe0ccd"}
Jul 31 12:15:16.354953 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.354924    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerStarted","Data":"48ddac0c5d4073e3512dbe43f48208a44bd5bfa1c7b4dde5b6271475f6a59c1c"}
Jul 31 12:15:16.356142 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.356117    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk" event={"ID":"46034009-6097-40de-bd1b-25569aa25011","Type":"ContainerStarted","Data":"4c29ab21f187455ba0f07d33109997327cb1905e568cb0afb28ff45840874408"}
Jul 31 12:15:16.357486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.357463    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pz4m5" event={"ID":"aa412f99-c8df-40db-9f64-a3e4122b5834","Type":"ContainerStarted","Data":"409eee29268233d73bd0983f28f0e1f5c12b6a01984e972f95bc42667fa2f5f8"}
Jul 31 12:15:16.358561 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.358537    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lkp6" event={"ID":"81c723fb-b45f-49ee-9a71-0f00ecb73e6b","Type":"ContainerStarted","Data":"ea37241cce700239d51e9d0392a5da2ae5db938c9ee32fa07a493b98f4b9b8be"}
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.791187    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.791242    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.791465    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.791528    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:18.791510259 +0000 UTC m=+6.023375012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.791913    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:16.792040 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.791964    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:18.791948505 +0000 UTC m=+6.023813238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:16.892417 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:16.892374    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:16.892567 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.892555    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:16.892638 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.892579    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:16.892638 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.892593    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:16.892742 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:16.892652    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:18.89263635 +0000 UTC m=+6.124501092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:17.207438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.207320    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 12:10:15 +0000 UTC" deadline="2028-01-05 17:26:33.014089003 +0000 UTC"
Jul 31 12:15:17.207438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.207381    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12557h11m15.806711454s"
Jul 31 12:15:17.343510 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.342876    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:17.343510 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:17.342994    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:17.343510 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.343376    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:17.343510 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:17.343458    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:17.383610 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.383578    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" event={"ID":"e7711a431e63758ca060bb86d740f9a6","Type":"ContainerStarted","Data":"973f9e7bc219c5a526225f41eab561158f72ca68fd4cd65b54c41c0a9d332a10"}
Jul 31 12:15:17.410015 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.409985    2532 generic.go:356] "Generic (PLEG): container finished" podID="c2321338cbfd1a0e5e9cde68af13d0e1" containerID="067d52c6af317cda13e01fdec6211771ffadfc2e4efada845d11ed1a2f52ece6" exitCode=0
Jul 31 12:15:17.410164 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.410031    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"c2321338cbfd1a0e5e9cde68af13d0e1","Type":"ContainerDied","Data":"067d52c6af317cda13e01fdec6211771ffadfc2e4efada845d11ed1a2f52ece6"}
Jul 31 12:15:17.424038 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:17.423987    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" podStartSLOduration=3.423968727 podStartE2EDuration="3.423968727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:17.398634823 +0000 UTC m=+4.630499575" watchObservedRunningTime="2026-07-31 12:15:17.423968727 +0000 UTC m=+4.655833479"
Jul 31 12:15:18.340271 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.340242    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:18.340623 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.340388    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:18.447197 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.443935    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"c2321338cbfd1a0e5e9cde68af13d0e1","Type":"ContainerStarted","Data":"dfd59086dfd8f66f9f8b032b5c7c823a45803af57e2b2fb299ca99a876e8b78f"}
Jul 31 12:15:18.462813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.462430    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" podStartSLOduration=4.462412372 podStartE2EDuration="4.462412372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:18.461723494 +0000 UTC m=+5.693588256" watchObservedRunningTime="2026-07-31 12:15:18.462412372 +0000 UTC m=+5.694277125"
Jul 31 12:15:18.806795 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.806755    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:18.806980 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.806809    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:18.806980 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.806928    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:18.807088 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.806991    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:22.806972572 +0000 UTC m=+10.038837325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:18.807428 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.807406    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:18.807499 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.807466    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:22.807449412 +0000 UTC m=+10.039314162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:18.907446 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:18.907411    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:18.907649 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.907629    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:18.907721 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.907654    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:18.907721 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.907668    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:18.907866 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:18.907729    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:22.907709631 +0000 UTC m=+10.139574383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:19.341196 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:19.339678    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:19.341196 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:19.339800    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:19.341196 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:19.339640    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:19.341196 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:19.340707    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:20.340730 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:20.340597    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:20.340897 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:20.340756    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:21.339913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:21.339712    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:21.339913 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:21.339827    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:21.340412 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:21.340243    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:21.340412 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:21.340335    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:22.340396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:22.340319    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:22.340898 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.340470    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:22.841103 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:22.841059    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:22.841311 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:22.841110    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:22.841311 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.841229    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:22.841311 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.841295    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:30.841277759 +0000 UTC m=+18.073142503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:22.841669 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.841229    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:22.841669 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.841641    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:30.841624411 +0000 UTC m=+18.073489168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:22.941749 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:22.941718    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:22.941966 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.941900    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:22.941966 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.941923    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:22.941966 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.941935    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:22.942160 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:22.941998    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:30.94197548 +0000 UTC m=+18.173840228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:23.341190 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:23.340581    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:23.341190 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:23.340682    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:23.341190 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:23.341049    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:23.341190 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:23.341137    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:24.339719 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:24.339687    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:24.339882 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:24.339835    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:25.340511 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:25.340481    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:25.340928 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:25.340603    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:25.340928 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:25.340676    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:25.340928 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:25.340765    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:26.340160 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:26.340128    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:26.340315 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:26.340289    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:27.340506 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:27.340472    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:27.340973 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:27.340596    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:27.340973 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:27.340655    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:27.340973 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:27.340767    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:28.339927 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:28.339893    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:28.340080 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:28.340014    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:29.340207 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:29.340156    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:29.340622 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:29.340165    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:29.340622 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:29.340282    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:29.340622 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:29.340378    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:30.340476 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:30.340446    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:30.340929 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:30.340559    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:30.905229 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:30.905190    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:30.905229 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:30.905230    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:30.908028 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:30.905692    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:30.908028 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:30.905834    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:15:46.905806544 +0000 UTC m=+34.137671289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:30.908028 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:30.906045    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:30.908028 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:30.906163    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:46.906147712 +0000 UTC m=+34.138012441 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:31.006331 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:31.006296    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:31.006485 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.006420    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:31.006485 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.006438    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:31.006485 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.006452    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:31.006639 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.006495    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:47.00648327 +0000 UTC m=+34.238348012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:31.340381 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:31.340345    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:31.340564 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:31.340388    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:31.340564 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.340478    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:31.340969 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:31.340593    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:32.340012 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:32.339978    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:32.340185 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:32.340092    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:33.340871 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:33.340816    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:33.341328 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:33.340935    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:33.341328 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:33.340953    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:33.341328 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:33.341057    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:34.340032 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.339655    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:34.340234 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:34.340158    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:34.475564 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.475536    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="87ed389ff43674420d9c4ef8efe47c10fd6998ce29401516d330a728477ce8df" exitCode=0
Jul 31 12:15:34.476313 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.475577    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"87ed389ff43674420d9c4ef8efe47c10fd6998ce29401516d330a728477ce8df"}
Jul 31 12:15:34.477210 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.477160    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk" event={"ID":"46034009-6097-40de-bd1b-25569aa25011","Type":"ContainerStarted","Data":"66c98ecb02f2a177107de618ec20255def483ec5e1a167c18c4f753e00b4e50d"}
Jul 31 12:15:34.478770 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.478739    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lkp6" event={"ID":"81c723fb-b45f-49ee-9a71-0f00ecb73e6b","Type":"ContainerStarted","Data":"7e9d3b33dd53fce9fc7a2368bf7e5f3b6e47f96176429af17739e60cd0c61038"}
Jul 31 12:15:34.480410 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.480386    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2x9rc" event={"ID":"fe0053fa-6869-42c1-8011-52836ccf5f5f","Type":"ContainerStarted","Data":"718ecb7d757588cd1b8dde0e2a56ea391bca64e9cfe00f68004e996fc4453bd8"}
Jul 31 12:15:34.481793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.481759    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zwgfj" event={"ID":"fd1c837c-3d3d-43f7-a61d-8a8774cd91cb","Type":"ContainerStarted","Data":"872b1a83d6b89ef519d3ddf657de3b6eaa7b20c25b773b868a03d1be28371369"}
Jul 31 12:15:34.484023 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.483999    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"30180f8ccb271123e545e4fc9bcae41149264bd3a1bc413fbc57e9fad06a8e8a"}
Jul 31 12:15:34.484101 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.484054    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"91d65cacfa558586eec2413e67baa6c5bef17732fc1696819aa2897218d37873"}
Jul 31 12:15:34.484149 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.484102    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"68fc9789a733835a9c6e4022c7c998ef2c4701b9ab5a310756ab76ce3d883f57"}
Jul 31 12:15:34.484201 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.484147    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"af482f48ef73b2a4ddce33374004a9a9112c26ec42efc6cb46546ea86bfa6323"}
Jul 31 12:15:34.485544 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.485522    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j6kr9" event={"ID":"a66df52f-4829-48d2-b940-6eba62a720b9","Type":"ContainerStarted","Data":"9568a923e6db8974154da24d38649c55b345bdad5b63ddbf0cc201290c92279e"}
Jul 31 12:15:34.486831 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.486801    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dgnbk" event={"ID":"50430d0c-78a9-4876-8dfe-9753de79bb9a","Type":"ContainerStarted","Data":"0f95e47fbb9eca2f8eb303f9086284d59d106a9aebee7de1cbc471c102a7a2b4"}
Jul 31 12:15:34.507810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.507771    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9lkp6" podStartSLOduration=3.811490472 podStartE2EDuration="21.507759886s" totalImagesPullingTime="17.696269414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:34.507567561 +0000 UTC m=+21.739432312" watchObservedRunningTime="2026-07-31 12:15:34.507759886 +0000 UTC m=+21.739624637"
Jul 31 12:15:34.522993 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.522945    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dgnbk" podStartSLOduration=3.822453794 podStartE2EDuration="21.522927231s" totalImagesPullingTime="17.700473437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:34.522492689 +0000 UTC m=+21.754357439" watchObservedRunningTime="2026-07-31 12:15:34.522927231 +0000 UTC m=+21.754791982"
Jul 31 12:15:34.536986 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.536948    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2x9rc" podStartSLOduration=3.837440409 podStartE2EDuration="21.536937386s" totalImagesPullingTime="17.699496977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:34.536638086 +0000 UTC m=+21.768502834" watchObservedRunningTime="2026-07-31 12:15:34.536937386 +0000 UTC m=+21.768802136"
Jul 31 12:15:34.551378 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.551337    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j6kr9" podStartSLOduration=8.076887694 podStartE2EDuration="21.551321937s" totalImagesPullingTime="13.474434243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:34.550952428 +0000 UTC m=+21.782817177" watchObservedRunningTime="2026-07-31 12:15:34.551321937 +0000 UTC m=+21.783186687"
Jul 31 12:15:34.601486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.601464    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:34.601912 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.601897    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:34.618917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:34.618882    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zwgfj" podStartSLOduration=3.869147985 podStartE2EDuration="21.618869052s" totalImagesPullingTime="17.749721067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:34.569086967 +0000 UTC m=+21.800951721" watchObservedRunningTime="2026-07-31 12:15:34.618869052 +0000 UTC m=+21.850733804"
Jul 31 12:15:35.340430 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.340397    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:35.340562 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:35.340524    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:35.341203 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.340981    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:35.341203 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:35.341074    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:35.422203 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.422165    2532 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 31 12:15:35.490969 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.490937    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk" event={"ID":"46034009-6097-40de-bd1b-25569aa25011","Type":"ContainerStarted","Data":"08fc055d818f1516b490b2de78b7f5677a637858c8071528dc626d7594582d91"}
Jul 31 12:15:35.493207 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.493160    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pz4m5" event={"ID":"aa412f99-c8df-40db-9f64-a3e4122b5834","Type":"ContainerStarted","Data":"8eb5aa1e6f313063f5715e67a8780fe7f8fd932a030a0ce7640ebbff4b886d23"}
Jul 31 12:15:35.496587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.496547    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"72aceb51abaddf2f0ed2f3909ee9e17cbd887f0ee3d03174159cbb0e94847921"}
Jul 31 12:15:35.496691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.496633    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"3e686333c63d5e36ad11796db78438007e70d776f95e56629e49b9276098f362"}
Jul 31 12:15:35.497410 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.497392    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:35.497873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.497856    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j6kr9"
Jul 31 12:15:35.520253 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:35.520200    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pz4m5" podStartSLOduration=4.828010043 podStartE2EDuration="22.520180883s" totalImagesPullingTime="17.69217084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:35.506824499 +0000 UTC m=+22.738689252" watchObservedRunningTime="2026-07-31 12:15:35.520180883 +0000 UTC m=+22.752045626"
Jul 31 12:15:36.222520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:36.222422    2532 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T12:15:35.422198986Z","UUID":"fe330e6e-4b38-4b5c-9def-348f15893e8d","Handler":null,"Name":"","Endpoint":""}
Jul 31 12:15:36.224014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:36.223988    2532 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 31 12:15:36.224134 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:36.224030    2532 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 31 12:15:36.340151 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:36.340121    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:36.340411 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:36.340260    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:37.340614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:37.340375    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:37.341026 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:37.340374    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:37.341026 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:37.340725    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:37.341026 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:37.340791    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:37.504395 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:37.504358    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"b8b66a1d35fbbec3224575d3277c44f0596ea23ed9747684f280cf2a50ede0e5"}
Jul 31 12:15:37.506803 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:37.506776    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk" event={"ID":"46034009-6097-40de-bd1b-25569aa25011","Type":"ContainerStarted","Data":"2fbbbf28f8f13751e1602e3cd912c2f52aaa2fd07e80e238f93be2320ea7ec1b"}
Jul 31 12:15:37.524476 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:37.524430    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqtgk" podStartSLOduration=4.690998914 podStartE2EDuration="24.52441959s" totalImagesPullingTime="19.833420676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:37.52426886 +0000 UTC m=+24.756133630" watchObservedRunningTime="2026-07-31 12:15:37.52441959 +0000 UTC m=+24.756284340"
Jul 31 12:15:38.340344 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:38.340315    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:38.340506 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:38.340453    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:39.339916 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:39.339876    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:39.340538 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:39.339874    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:39.340538 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:39.340001    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:39.340538 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:39.340086    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:40.340336 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.340147    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:40.340796 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:40.340425    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:40.513815 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.513774    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="7b1707e7d229586908d250b94cae6eec05fc2e93d49f1a3d96dad0183a35ab47" exitCode=0
Jul 31 12:15:40.514015 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.513851    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"7b1707e7d229586908d250b94cae6eec05fc2e93d49f1a3d96dad0183a35ab47"}
Jul 31 12:15:40.517554 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.517531    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" event={"ID":"2d8617f0-21ff-447c-a82c-176a4d647a9b","Type":"ContainerStarted","Data":"964832327f94fc2c38d2dce0a2ae09d9efa413f4bd9113bc69ccf4d441c08047"}
Jul 31 12:15:40.517853 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.517805    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:40.517958 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.517874    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:40.532746 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.532726    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:40.532961 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.532925    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:40.557209 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:40.557155    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b" podStartSLOduration=9.799180562 podStartE2EDuration="27.557144547s" totalImagesPullingTime="17.757963985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:40.557021573 +0000 UTC m=+27.788886323" watchObservedRunningTime="2026-07-31 12:15:40.557144547 +0000 UTC m=+27.789009298"
Jul 31 12:15:41.340436 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.340406    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:41.340904 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:41.340548    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:41.340904 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.340593    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:41.340904 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:41.340715    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:41.523107 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.523082    2532 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 31 12:15:41.523231 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.523133    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerStarted","Data":"44e47e8eb44b92d41bbbf31e4ee49f69e1f123fa0debc442e9522738aef8afe7"}
Jul 31 12:15:41.669036 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.668860    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqkmw"]
Jul 31 12:15:41.669134 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.669111    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:41.669239 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:41.669221    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.673681    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59h74"]
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.674076    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:41.674240    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.674938    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jmffq"]
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:41.675048    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:41.676029 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:41.675196    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:42.526514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:42.526482    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="44e47e8eb44b92d41bbbf31e4ee49f69e1f123fa0debc442e9522738aef8afe7" exitCode=0
Jul 31 12:15:42.526908 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:42.526571    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"44e47e8eb44b92d41bbbf31e4ee49f69e1f123fa0debc442e9522738aef8afe7"}
Jul 31 12:15:42.526908 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:42.526637    2532 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 31 12:15:43.234343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.233787    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:15:43.340685 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.340665    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:43.340784 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:43.340763    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:43.340842 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.340786    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:43.340929 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:43.340910    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:43.340995 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.340979    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:43.341104 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:43.341082    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:43.531333 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.531247    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="899755d1bc28c095240440e2635ed793e67d9c43da3a3ba410eef93be8397521" exitCode=0
Jul 31 12:15:43.531927 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:43.531335    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"899755d1bc28c095240440e2635ed793e67d9c43da3a3ba410eef93be8397521"}
Jul 31 12:15:45.340161 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.340128    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:45.340553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.340133    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:45.340553 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.340244    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmffq" podUID="958febfc-8fca-4277-8826-2c43d602d2e8"
Jul 31 12:15:45.340553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.340133    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:45.340553 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.340311    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:15:45.340553 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.340368    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqkmw" podUID="01539827-8873-463d-89b1-ff47d4e4b2a8"
Jul 31 12:15:45.559420 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.559400    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeReady"
Jul 31 12:15:45.559519 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.559502    2532 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 31 12:15:45.591273 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.591217    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:15:45.603365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.603344    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.606141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.605567    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:15:45.606141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.606061    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 31 12:15:45.606316 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.606151    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 31 12:15:45.606316 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.606250    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8wz22"]
Jul 31 12:15:45.606970 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.606508    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 31 12:15:45.606970 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.606595    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tjczm\""
Jul 31 12:15:45.608608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.608588    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.610819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.610799    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zlkhg"]
Jul 31 12:15:45.610952 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.610931    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 31 12:15:45.611370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.611319    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 31 12:15:45.611495 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.611471    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ngjpv\""
Jul 31 12:15:45.613560 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.613534    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.615642 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.615620    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 31 12:15:45.616414 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.616218    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 31 12:15:45.616414 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.616372    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 31 12:15:45.616733 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.616706    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4h696\""
Jul 31 12:15:45.617033 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.616817    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 31 12:15:45.620007 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.619988    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wz22"]
Jul 31 12:15:45.621796 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.621769    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zlkhg"]
Jul 31 12:15:45.712806 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712740    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm8k7\" (UniqueName: \"kubernetes.io/projected/de449ca7-7464-4981-825f-9c3fc3dead90-kube-api-access-rm8k7\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.712929 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712851    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.712929 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712898    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.712929 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712923    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6bk2\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712951    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712973    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.712992    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713013    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld9r6\" (UniqueName: \"kubernetes.io/projected/1107bc35-a8da-4171-a247-271408b87167-kube-api-access-ld9r6\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713031    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1107bc35-a8da-4171-a247-271408b87167-cert\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713051    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.713067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713068    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de449ca7-7464-4981-825f-9c3fc3dead90-tmp-dir\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.713370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713085    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713123    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.713370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.713201    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de449ca7-7464-4981-825f-9c3fc3dead90-config-volume\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.814052 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814016    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814052 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814056    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814076    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814107    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld9r6\" (UniqueName: \"kubernetes.io/projected/1107bc35-a8da-4171-a247-271408b87167-kube-api-access-ld9r6\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814130    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1107bc35-a8da-4171-a247-271408b87167-cert\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814198    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.814217    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.814236    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814238    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de449ca7-7464-4981-825f-9c3fc3dead90-tmp-dir\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.814268 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814266    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.814308    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:15:46.314288456 +0000 UTC m=+33.546153209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814330    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814367    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de449ca7-7464-4981-825f-9c3fc3dead90-config-volume\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.814374    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814416    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm8k7\" (UniqueName: \"kubernetes.io/projected/de449ca7-7464-4981-825f-9c3fc3dead90-kube-api-access-rm8k7\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:45.814439    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:46.314428827 +0000 UTC m=+33.546293571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814474    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814508    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814529    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6bk2\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.814683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.814572    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de449ca7-7464-4981-825f-9c3fc3dead90-tmp-dir\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.815143 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.815037    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.815143 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.815088    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.815239 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.815139    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de449ca7-7464-4981-825f-9c3fc3dead90-config-volume\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.815321 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.815303    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.818287 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.818267    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.818384 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.818273    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1107bc35-a8da-4171-a247-271408b87167-cert\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.818384 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.818337    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.825730 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.825704    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6bk2\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.825730 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.825717    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:45.826285 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.826264    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm8k7\" (UniqueName: \"kubernetes.io/projected/de449ca7-7464-4981-825f-9c3fc3dead90-kube-api-access-rm8k7\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:45.826384 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.826370    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld9r6\" (UniqueName: \"kubernetes.io/projected/1107bc35-a8da-4171-a247-271408b87167-kube-api-access-ld9r6\") pod \"ingress-canary-zlkhg\" (UID: \"1107bc35-a8da-4171-a247-271408b87167\") " pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:45.934637 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:45.934579    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zlkhg"
Jul 31 12:15:46.159262 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.159032    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zlkhg"]
Jul 31 12:15:46.162184 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:15:46.162147    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1107bc35_a8da_4171_a247_271408b87167.slice/crio-87b8c1afb30ee35f07eb3d4970ccf09df98716fdf69adc9e5f79908cc6e3c3d5 WatchSource:0}: Error finding container 87b8c1afb30ee35f07eb3d4970ccf09df98716fdf69adc9e5f79908cc6e3c3d5: Status 404 returned error can't find the container with id 87b8c1afb30ee35f07eb3d4970ccf09df98716fdf69adc9e5f79908cc6e3c3d5
Jul 31 12:15:46.318162 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.318138    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:46.318287 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.318209    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:46.318287 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.318266    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:15:46.318287 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.318281    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:15:46.318381 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.318309    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:15:46.318381 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.318327    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:15:47.318313814 +0000 UTC m=+34.550178543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:15:46.318381 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.318356    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:47.318345434 +0000 UTC m=+34.550210163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:15:46.538859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.538831    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zlkhg" event={"ID":"1107bc35-a8da-4171-a247-271408b87167","Type":"ContainerStarted","Data":"87b8c1afb30ee35f07eb3d4970ccf09df98716fdf69adc9e5f79908cc6e3c3d5"}
Jul 31 12:15:46.921929 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.921902    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:46.922212 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:46.921938    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:46.922212 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.922052    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:46.922212 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.922086    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:46.922212 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.922116    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret podName:01539827-8873-463d-89b1-ff47d4e4b2a8 nodeName:}" failed. No retries permitted until 2026-07-31 12:16:18.92209958 +0000 UTC m=+66.153964332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret") pod "global-pull-secret-syncer-xqkmw" (UID: "01539827-8873-463d-89b1-ff47d4e4b2a8") : object "kube-system"/"original-pull-secret" not registered
Jul 31 12:15:46.922212 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:46.922147    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:16:18.922129643 +0000 UTC m=+66.153994402 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 12:15:47.022358 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.022325    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:47.022505 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.022488    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 12:15:47.022505 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.022502    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 12:15:47.022587 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.022512    2532 projected.go:196] Error preparing data for projected volume kube-api-access-498qb for pod openshift-network-diagnostics/network-check-target-jmffq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:47.022587 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.022564    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb podName:958febfc-8fca-4277-8826-2c43d602d2e8 nodeName:}" failed. No retries permitted until 2026-07-31 12:16:19.022550978 +0000 UTC m=+66.254415710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-498qb" (UniqueName: "kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb") pod "network-check-target-jmffq" (UID: "958febfc-8fca-4277-8826-2c43d602d2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 12:15:47.324555 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.324517    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:47.324723 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.324659    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:15:47.324723 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.324692    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:47.324808 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.324729    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:49.324708271 +0000 UTC m=+36.556573001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:15:47.324808 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.324786    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:15:47.324808 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.324799    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:15:47.324939 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:47.324845    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:15:49.324829516 +0000 UTC m=+36.556694262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:15:47.340370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.340345    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:15:47.340370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.340359    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:15:47.340519 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.340345    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:15:47.343545 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343527    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dw9nv\""
Jul 31 12:15:47.343670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343563    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 12:15:47.343670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343527    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qqfkp\""
Jul 31 12:15:47.343820 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343803    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 12:15:47.343884 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343826    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 12:15:47.343884 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:47.343826    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 12:15:49.341905 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:49.341867    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:49.343029 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:49.342992    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:49.343156 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:49.343106    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:15:49.343156 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:49.343130    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:15:49.343156 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:49.343141    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:15:49.343326 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:49.343231    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:15:53.343194963 +0000 UTC m=+40.575059709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:15:49.343326 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:49.343268    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:15:53.343253437 +0000 UTC m=+40.575118168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:15:50.550320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:50.550079    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zlkhg" event={"ID":"1107bc35-a8da-4171-a247-271408b87167","Type":"ContainerStarted","Data":"bbbd6da1b205a5eadcafd64e20a3d4ce9c0dd86a0b93fbb33dcac7b3f53e78b8"}
Jul 31 12:15:50.552724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:50.552698    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="14d5e70f7bdd018f3cdea11eff0dc56545352ddd28417582d6ca4b0e984ddaa0" exitCode=0
Jul 31 12:15:50.552825 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:50.552744    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"14d5e70f7bdd018f3cdea11eff0dc56545352ddd28417582d6ca4b0e984ddaa0"}
Jul 31 12:15:50.586786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:50.586739    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zlkhg" podStartSLOduration=1.810031553 podStartE2EDuration="5.586720487s" totalImagesPullingTime="3.776688934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:50.565420678 +0000 UTC m=+37.797285431" watchObservedRunningTime="2026-07-31 12:15:50.586720487 +0000 UTC m=+37.818585237"
Jul 31 12:15:51.557099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:51.557028    2532 generic.go:356] "Generic (PLEG): container finished" podID="49bfc914-42f7-4768-9448-20c7ce0121ce" containerID="ec342b6428024b1d47020961cb220a65668e6f23b7d0fd75e68cedfac5f0a9c8" exitCode=0
Jul 31 12:15:51.557549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:51.557108    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerDied","Data":"ec342b6428024b1d47020961cb220a65668e6f23b7d0fd75e68cedfac5f0a9c8"}
Jul 31 12:15:52.561921 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:52.561887    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t775z" event={"ID":"49bfc914-42f7-4768-9448-20c7ce0121ce","Type":"ContainerStarted","Data":"dcdc346886bd8cf95f0d467b9b57b5bb9c74ec073b165578923f30183bd60fb9"}
Jul 31 12:15:52.582767 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:52.582719    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t775z" podStartSLOduration=8.387031802 podStartE2EDuration="39.582704339s" totalImagesPullingTime="31.195672537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:15:52.581497754 +0000 UTC m=+39.813362504" watchObservedRunningTime="2026-07-31 12:15:52.582704339 +0000 UTC m=+39.814569068"
Jul 31 12:15:53.373870 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:53.373844    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:15:53.373886    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:53.373977    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:53.373980    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:53.373994    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:53.374020    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:16:01.374008423 +0000 UTC m=+48.605873152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:15:53.374037 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:15:53.374032    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:16:01.374026768 +0000 UTC m=+48.605891497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:16:01.431053 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:01.431019    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:01.431100    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:01.431159    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:01.431204    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:01.431214    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:01.431230    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:16:17.431214869 +0000 UTC m=+64.663079621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:16:01.431435 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:01.431248    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:16:17.431237782 +0000 UTC m=+64.663102535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:16:13.541160 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:13.541138    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lw6b"
Jul 31 12:16:17.435313 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:17.435277    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:16:17.435313 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:17.435322    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:16:17.435717 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:17.435423    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:16:17.435717 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:17.435426    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:16:17.435717 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:17.435445    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:16:17.435717 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:17.435478    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:16:49.435463359 +0000 UTC m=+96.667328090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:16:17.435717 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:17.435491    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:16:49.435485588 +0000 UTC m=+96.667350317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:16:18.944373 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:18.944335    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:16:18.944373 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:18.944375    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:16:18.950253 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:18.950233    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 12:16:18.950253 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:18.950245    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 12:16:18.955065 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:18.955049    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 31 12:16:18.955118 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:18.955108    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:17:22.955093442 +0000 UTC m=+130.186958171 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : secret "metrics-daemon-secret" not found
Jul 31 12:16:18.958904 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:18.958873    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01539827-8873-463d-89b1-ff47d4e4b2a8-original-pull-secret\") pod \"global-pull-secret-syncer-xqkmw\" (UID: \"01539827-8873-463d-89b1-ff47d4e4b2a8\") " pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:16:19.044832 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.044810    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:16:19.047177 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.047153    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 12:16:19.057611 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.057596    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 12:16:19.067266 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.067247    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-498qb\" (UniqueName: \"kubernetes.io/projected/958febfc-8fca-4277-8826-2c43d602d2e8-kube-api-access-498qb\") pod \"network-check-target-jmffq\" (UID: \"958febfc-8fca-4277-8826-2c43d602d2e8\") " pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:16:19.159803 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.159780    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dw9nv\""
Jul 31 12:16:19.162436 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.162423    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqkmw"
Jul 31 12:16:19.168114 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.168100    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:16:19.292162 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.292042    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqkmw"]
Jul 31 12:16:19.295605 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:16:19.295583    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01539827_8873_463d_89b1_ff47d4e4b2a8.slice/crio-ca8de9f681faa2ceb103da7291acc1a70e83e08f3de59f26abc9847dfc4b7ea1 WatchSource:0}: Error finding container ca8de9f681faa2ceb103da7291acc1a70e83e08f3de59f26abc9847dfc4b7ea1: Status 404 returned error can't find the container with id ca8de9f681faa2ceb103da7291acc1a70e83e08f3de59f26abc9847dfc4b7ea1
Jul 31 12:16:19.305149 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.305127    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jmffq"]
Jul 31 12:16:19.307613 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:16:19.307589    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958febfc_8fca_4277_8826_2c43d602d2e8.slice/crio-0cda0f74e57b3da04c1ad7077f6c09a30d81d2b34859ee4b7ba186b89f98c422 WatchSource:0}: Error finding container 0cda0f74e57b3da04c1ad7077f6c09a30d81d2b34859ee4b7ba186b89f98c422: Status 404 returned error can't find the container with id 0cda0f74e57b3da04c1ad7077f6c09a30d81d2b34859ee4b7ba186b89f98c422
Jul 31 12:16:19.621190 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.621100    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jmffq" event={"ID":"958febfc-8fca-4277-8826-2c43d602d2e8","Type":"ContainerStarted","Data":"0cda0f74e57b3da04c1ad7077f6c09a30d81d2b34859ee4b7ba186b89f98c422"}
Jul 31 12:16:19.622093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:19.622075    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqkmw" event={"ID":"01539827-8873-463d-89b1-ff47d4e4b2a8","Type":"ContainerStarted","Data":"ca8de9f681faa2ceb103da7291acc1a70e83e08f3de59f26abc9847dfc4b7ea1"}
Jul 31 12:16:24.636509 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:24.636430    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jmffq" event={"ID":"958febfc-8fca-4277-8826-2c43d602d2e8","Type":"ContainerStarted","Data":"dc335b8af641f85ce626460f1f6f27c03739e865f77538ac72a730d789230d66"}
Jul 31 12:16:24.636902 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:24.636534    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:16:24.637895 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:24.637871    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqkmw" event={"ID":"01539827-8873-463d-89b1-ff47d4e4b2a8","Type":"ContainerStarted","Data":"cd214a6742c0d23716852bd2e61d2831220b840944b0a2421c2b871cb7723637"}
Jul 31 12:16:24.650987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:24.650950    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jmffq" podStartSLOduration=66.657754064 podStartE2EDuration="1m11.650940336s" totalImagesPullingTime="4.993186272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:16:24.650513975 +0000 UTC m=+71.882378727" watchObservedRunningTime="2026-07-31 12:16:24.650940336 +0000 UTC m=+71.882805086"
Jul 31 12:16:24.664069 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:24.664032    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xqkmw" podStartSLOduration=66.666947319 podStartE2EDuration="1m11.664020903s" totalImagesPullingTime="4.997073584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:16:24.663286599 +0000 UTC m=+71.895151374" watchObservedRunningTime="2026-07-31 12:16:24.664020903 +0000 UTC m=+71.895885669"
Jul 31 12:16:49.447475 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:49.447347    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:16:49.447475 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:49.447417    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:16:49.448047 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:49.447517    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:16:49.448047 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:49.447532    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:16:49.448047 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:49.447517    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:16:49.448047 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:49.447602    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:17:53.447580077 +0000 UTC m=+160.679444808 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:16:49.448047 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:16:49.447656    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:17:53.447638507 +0000 UTC m=+160.679503253 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:16:55.642548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:16:55.642503    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jmffq"
Jul 31 12:17:22.966081 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:17:22.966034    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:17:22.966473 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:22.966180    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 31 12:17:22.966473 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:22.966245    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:19:24.966229417 +0000 UTC m=+252.198094146 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : secret "metrics-daemon-secret" not found
Jul 31 12:17:48.620326 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:48.620280    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b"
Jul 31 12:17:48.629424 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:48.629403    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wz22" podUID="de449ca7-7464-4981-825f-9c3fc3dead90"
Jul 31 12:17:48.829358 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:17:48.829332    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:17:48.829471 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:17:48.829367    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:17:50.351152 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:50.351105    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:17:53.465558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:17:53.465525    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:17:53.465567    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:53.465657    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:53.465661    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:53.465675    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b9bc94b4-jcqjv: secret "image-registry-tls" not found
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:53.465711    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls podName:de449ca7-7464-4981-825f-9c3fc3dead90 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:55.465697549 +0000 UTC m=+282.697562291 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls") pod "dns-default-8wz22" (UID: "de449ca7-7464-4981-825f-9c3fc3dead90") : secret "dns-default-metrics-tls" not found
Jul 31 12:17:53.465943 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:17:53.465723    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls podName:38eb5134-0c8b-4b1d-868f-2eecaf7c827b nodeName:}" failed. No retries permitted until 2026-07-31 12:19:55.465717345 +0000 UTC m=+282.697582074 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls") pod "image-registry-77b9bc94b4-jcqjv" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b") : secret "image-registry-tls" not found
Jul 31 12:18:05.340288 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:18:05.340198    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:19:23.870512 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.870482    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"]
Jul 31 12:19:23.872256 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.872241    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:23.876743 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.876720    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 31 12:19:23.876843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.876813    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 31 12:19:23.876910 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.876869    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 31 12:19:23.877413 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.877389    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:19:23.877536 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.877522    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sggkr\""
Jul 31 12:19:23.880998 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.880980    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"]
Jul 31 12:19:23.882702 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.882687    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-vgntt"]
Jul 31 12:19:23.882817 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.882801    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:23.884520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.884501    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"]
Jul 31 12:19:23.884597 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.884579    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.886919 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.886904    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 31 12:19:23.887000 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.886975    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 31 12:19:23.887295 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887279    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 31 12:19:23.887488 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887296    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 31 12:19:23.887683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887306    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 31 12:19:23.887683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887622    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 31 12:19:23.887683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887641    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 31 12:19:23.887898 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887755    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-n7m8d\""
Jul 31 12:19:23.888016 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.887994    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 31 12:19:23.888235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.888209    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fm2lg\""
Jul 31 12:19:23.893150 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.893128    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 31 12:19:23.903214 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.903194    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-vgntt"]
Jul 31 12:19:23.903921 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.903901    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"]
Jul 31 12:19:23.934898 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.934873    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-serving-cert\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935035 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.934907    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-service-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935035 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.934931    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h89mw\" (UniqueName: \"kubernetes.io/projected/bdb35b15-fb85-4389-9a66-7ba41b50e853-kube-api-access-h89mw\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:23.935035 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.934955    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdb35b15-fb85-4389-9a66-7ba41b50e853-config\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:23.935035 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935018    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e553f719-737d-46b1-b8e7-b58473ee0a60-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:23.935195 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935056    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdb35b15-fb85-4389-9a66-7ba41b50e853-serving-cert\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:23.935195 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935089    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-tmp\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935195 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935104    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-snapshots\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935195 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935119    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqk6h\" (UniqueName: \"kubernetes.io/projected/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-kube-api-access-hqk6h\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935195 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935166    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-trusted-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:23.935348 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935206    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdtc\" (UniqueName: \"kubernetes.io/projected/e553f719-737d-46b1-b8e7-b58473ee0a60-kube-api-access-ffdtc\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:23.935348 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.935228    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:23.965184 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.965146    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"]
Jul 31 12:19:23.966902 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.966887    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"
Jul 31 12:19:23.968809 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.968788    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"]
Jul 31 12:19:23.969353 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.969332    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 31 12:19:23.969444 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.969387    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cdtww\""
Jul 31 12:19:23.969624 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.969604    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:19:23.970641 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.970625    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"]
Jul 31 12:19:23.970770 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.970748    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:23.972483 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.972455    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"
Jul 31 12:19:23.973028 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.972980    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qmfkk\""
Jul 31 12:19:23.973132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.973062    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 31 12:19:23.973486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.973459    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:19:23.974048 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.974031    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 31 12:19:23.974608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.974592    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 31 12:19:23.976492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.976474    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mm9bv\""
Jul 31 12:19:23.977593 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.977569    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"]
Jul 31 12:19:23.984762 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.984744    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"]
Jul 31 12:19:23.989336 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:23.989318    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"]
Jul 31 12:19:24.035740 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035717    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e553f719-737d-46b1-b8e7-b58473ee0a60-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.035836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035746    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdb35b15-fb85-4389-9a66-7ba41b50e853-serving-cert\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.035836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035768    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-snapshots\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.035836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035799    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h89mw\" (UniqueName: \"kubernetes.io/projected/bdb35b15-fb85-4389-9a66-7ba41b50e853-kube-api-access-h89mw\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.035982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035870    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-serving-cert\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.035982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035910    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47fwx\" (UniqueName: \"kubernetes.io/projected/a43a9396-e69f-4194-bddb-e0125e2013d8-kube-api-access-47fwx\") pod \"network-check-source-65bdbbff45-7gxbk\" (UID: \"a43a9396-e69f-4194-bddb-e0125e2013d8\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"
Jul 31 12:19:24.035982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.035941    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8562adee-da47-4daf-8786-144e50002130-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.036097 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036056    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqk6h\" (UniqueName: \"kubernetes.io/projected/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-kube-api-access-hqk6h\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036097 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036086    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-trusted-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036224 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036113    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.036224 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036139    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-tmp\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036224 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036205    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdtc\" (UniqueName: \"kubernetes.io/projected/e553f719-737d-46b1-b8e7-b58473ee0a60-kube-api-access-ffdtc\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.036352 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.036242    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:24.036352 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036244    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkt9k\" (UniqueName: \"kubernetes.io/projected/d7843221-f183-4ddd-b2d4-e4ded5d90b8d-kube-api-access-dkt9k\") pod \"volume-data-source-validator-5b8776669-8rvn4\" (UID: \"d7843221-f183-4ddd-b2d4-e4ded5d90b8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"
Jul 31 12:19:24.036352 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.036335    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:24.536313064 +0000 UTC m=+251.768177809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:24.036492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036379    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vgp2\" (UniqueName: \"kubernetes.io/projected/8562adee-da47-4daf-8786-144e50002130-kube-api-access-2vgp2\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.036492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036413    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-service-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036440    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8562adee-da47-4daf-8786-144e50002130-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.036492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036458    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-snapshots\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036687 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036466    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdb35b15-fb85-4389-9a66-7ba41b50e853-config\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.036687 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036503    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-tmp\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.036687 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036528    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e553f719-737d-46b1-b8e7-b58473ee0a60-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.036983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.036961    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-service-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.037060 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.037017    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-trusted-ca-bundle\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.037114 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.037057    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdb35b15-fb85-4389-9a66-7ba41b50e853-config\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.038353 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.038337    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-serving-cert\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.038481 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.038467    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdb35b15-fb85-4389-9a66-7ba41b50e853-serving-cert\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.042702 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.042680    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h89mw\" (UniqueName: \"kubernetes.io/projected/bdb35b15-fb85-4389-9a66-7ba41b50e853-kube-api-access-h89mw\") pod \"service-ca-operator-784f97bcf6-58hvb\" (UID: \"bdb35b15-fb85-4389-9a66-7ba41b50e853\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.045673 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.045647    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdtc\" (UniqueName: \"kubernetes.io/projected/e553f719-737d-46b1-b8e7-b58473ee0a60-kube-api-access-ffdtc\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.045878 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.045860    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqk6h\" (UniqueName: \"kubernetes.io/projected/8bd46abd-03c1-4f7d-88d0-8957be2aeb8e-kube-api-access-hqk6h\") pod \"insights-operator-77696685b-vgntt\" (UID: \"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e\") " pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.065266 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.065248    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"]
Jul 31 12:19:24.067498 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.067485    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.069670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.069653    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 31 12:19:24.069765 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.069749    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 31 12:19:24.069896 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.069878    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:19:24.070002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.069987    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gw77p\""
Jul 31 12:19:24.078011 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.077988    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"]
Jul 31 12:19:24.136968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.136917    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.136968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.136946    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47fwx\" (UniqueName: \"kubernetes.io/projected/a43a9396-e69f-4194-bddb-e0125e2013d8-kube-api-access-47fwx\") pod \"network-check-source-65bdbbff45-7gxbk\" (UID: \"a43a9396-e69f-4194-bddb-e0125e2013d8\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"
Jul 31 12:19:24.136968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.136965    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8562adee-da47-4daf-8786-144e50002130-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.137163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.137062    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvzbq\" (UniqueName: \"kubernetes.io/projected/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-kube-api-access-jvzbq\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.137163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.137103    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkt9k\" (UniqueName: \"kubernetes.io/projected/d7843221-f183-4ddd-b2d4-e4ded5d90b8d-kube-api-access-dkt9k\") pod \"volume-data-source-validator-5b8776669-8rvn4\" (UID: \"d7843221-f183-4ddd-b2d4-e4ded5d90b8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"
Jul 31 12:19:24.137163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.137132    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vgp2\" (UniqueName: \"kubernetes.io/projected/8562adee-da47-4daf-8786-144e50002130-kube-api-access-2vgp2\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.137163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.137160    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8562adee-da47-4daf-8786-144e50002130-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.137619 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.137596    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8562adee-da47-4daf-8786-144e50002130-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.138836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.138815    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8562adee-da47-4daf-8786-144e50002130-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.144225 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.144205    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vgp2\" (UniqueName: \"kubernetes.io/projected/8562adee-da47-4daf-8786-144e50002130-kube-api-access-2vgp2\") pod \"kube-storage-version-migrator-operator-7878f4b88c-sbf7v\" (UID: \"8562adee-da47-4daf-8786-144e50002130\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.144530 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.144512    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkt9k\" (UniqueName: \"kubernetes.io/projected/d7843221-f183-4ddd-b2d4-e4ded5d90b8d-kube-api-access-dkt9k\") pod \"volume-data-source-validator-5b8776669-8rvn4\" (UID: \"d7843221-f183-4ddd-b2d4-e4ded5d90b8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"
Jul 31 12:19:24.144671 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.144651    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47fwx\" (UniqueName: \"kubernetes.io/projected/a43a9396-e69f-4194-bddb-e0125e2013d8-kube-api-access-47fwx\") pod \"network-check-source-65bdbbff45-7gxbk\" (UID: \"a43a9396-e69f-4194-bddb-e0125e2013d8\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"
Jul 31 12:19:24.180251 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.180232    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"
Jul 31 12:19:24.200508 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.200487    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-vgntt"
Jul 31 12:19:24.238653 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.237961    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.238653 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.238062    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvzbq\" (UniqueName: \"kubernetes.io/projected/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-kube-api-access-jvzbq\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.239139 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.238876    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 12:19:24.239139 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.238958    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls podName:aa767e9c-0cfc-4f2c-93d5-15da50d670c8 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:24.738936855 +0000 UTC m=+251.970801604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jbl62" (UID: "aa767e9c-0cfc-4f2c-93d5-15da50d670c8") : secret "samples-operator-tls" not found
Jul 31 12:19:24.247009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.246977    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvzbq\" (UniqueName: \"kubernetes.io/projected/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-kube-api-access-jvzbq\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.276596 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.276571    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"
Jul 31 12:19:24.282773 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.282746    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"
Jul 31 12:19:24.288104 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.288084    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"
Jul 31 12:19:24.306766 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.306673    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb"]
Jul 31 12:19:24.311884 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:24.311835    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb35b15_fb85_4389_9a66_7ba41b50e853.slice/crio-51471af864c046eaf8e70fbbff90a7a6a733b1430e0a57653ec5f7528bcdae1d WatchSource:0}: Error finding container 51471af864c046eaf8e70fbbff90a7a6a733b1430e0a57653ec5f7528bcdae1d: Status 404 returned error can't find the container with id 51471af864c046eaf8e70fbbff90a7a6a733b1430e0a57653ec5f7528bcdae1d
Jul 31 12:19:24.327193 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.327001    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-vgntt"]
Jul 31 12:19:24.330074 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:24.330040    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd46abd_03c1_4f7d_88d0_8957be2aeb8e.slice/crio-7fa2f887a4877c46ed3b4ebeb789e3889f161d442fe40846786904acbc16cb46 WatchSource:0}: Error finding container 7fa2f887a4877c46ed3b4ebeb789e3889f161d442fe40846786904acbc16cb46: Status 404 returned error can't find the container with id 7fa2f887a4877c46ed3b4ebeb789e3889f161d442fe40846786904acbc16cb46
Jul 31 12:19:24.418852 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.418772    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4"]
Jul 31 12:19:24.421767 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:24.421392    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7843221_f183_4ddd_b2d4_e4ded5d90b8d.slice/crio-f734f6927daa19f068b66de51321ea6a73b065d337303a101566a4dd891d1d91 WatchSource:0}: Error finding container f734f6927daa19f068b66de51321ea6a73b065d337303a101566a4dd891d1d91: Status 404 returned error can't find the container with id f734f6927daa19f068b66de51321ea6a73b065d337303a101566a4dd891d1d91
Jul 31 12:19:24.433584 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.433551    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v"]
Jul 31 12:19:24.436641 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:24.436618    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8562adee_da47_4daf_8786_144e50002130.slice/crio-3dfd3c6a8ac86800c8552618f9bacad9d05345855c665887eb30a620d73eb547 WatchSource:0}: Error finding container 3dfd3c6a8ac86800c8552618f9bacad9d05345855c665887eb30a620d73eb547: Status 404 returned error can't find the container with id 3dfd3c6a8ac86800c8552618f9bacad9d05345855c665887eb30a620d73eb547
Jul 31 12:19:24.447453 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.447429    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk"]
Jul 31 12:19:24.449477 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:24.449452    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43a9396_e69f_4194_bddb_e0125e2013d8.slice/crio-f69637adcb658ba20f6416f6abc5e20f7cdcd56394bbecfbb7ba0f78dd6d31d1 WatchSource:0}: Error finding container f69637adcb658ba20f6416f6abc5e20f7cdcd56394bbecfbb7ba0f78dd6d31d1: Status 404 returned error can't find the container with id f69637adcb658ba20f6416f6abc5e20f7cdcd56394bbecfbb7ba0f78dd6d31d1
Jul 31 12:19:24.540378 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.540341    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:24.540509 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.540477    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:24.540546 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.540534    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:25.540520232 +0000 UTC m=+252.772384961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:24.741640 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:24.741604    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:24.741796 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.741767    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 12:19:24.741872 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:24.741847    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls podName:aa767e9c-0cfc-4f2c-93d5-15da50d670c8 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:25.741825082 +0000 UTC m=+252.973689833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jbl62" (UID: "aa767e9c-0cfc-4f2c-93d5-15da50d670c8") : secret "samples-operator-tls" not found
Jul 31 12:19:25.045559 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.045484    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:19:25.045972 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.045675    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 31 12:19:25.045972 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.045737    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs podName:f581f940-84e7-41b3-a723-29e3e33fceac nodeName:}" failed. No retries permitted until 2026-07-31 12:21:27.04571809 +0000 UTC m=+374.277582821 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs") pod "network-metrics-daemon-59h74" (UID: "f581f940-84e7-41b3-a723-29e3e33fceac") : secret "metrics-daemon-secret" not found
Jul 31 12:19:25.046093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.045981    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk" event={"ID":"a43a9396-e69f-4194-bddb-e0125e2013d8","Type":"ContainerStarted","Data":"0eb781148a50cffc71fedbbf88218bd0b2de46c6a70a6920ef190d155348b117"}
Jul 31 12:19:25.046093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.046028    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk" event={"ID":"a43a9396-e69f-4194-bddb-e0125e2013d8","Type":"ContainerStarted","Data":"f69637adcb658ba20f6416f6abc5e20f7cdcd56394bbecfbb7ba0f78dd6d31d1"}
Jul 31 12:19:25.047276 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.047252    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-vgntt" event={"ID":"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e","Type":"ContainerStarted","Data":"7fa2f887a4877c46ed3b4ebeb789e3889f161d442fe40846786904acbc16cb46"}
Jul 31 12:19:25.048761 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.048734    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb" event={"ID":"bdb35b15-fb85-4389-9a66-7ba41b50e853","Type":"ContainerStarted","Data":"51471af864c046eaf8e70fbbff90a7a6a733b1430e0a57653ec5f7528bcdae1d"}
Jul 31 12:19:25.051340 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.050998    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4" event={"ID":"d7843221-f183-4ddd-b2d4-e4ded5d90b8d","Type":"ContainerStarted","Data":"f734f6927daa19f068b66de51321ea6a73b065d337303a101566a4dd891d1d91"}
Jul 31 12:19:25.052576 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.052519    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v" event={"ID":"8562adee-da47-4daf-8786-144e50002130","Type":"ContainerStarted","Data":"3dfd3c6a8ac86800c8552618f9bacad9d05345855c665887eb30a620d73eb547"}
Jul 31 12:19:25.066259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.065497    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-7gxbk" podStartSLOduration=2.065476835 podStartE2EDuration="2.065476835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:25.060766176 +0000 UTC m=+252.292630927" watchObservedRunningTime="2026-07-31 12:19:25.065476835 +0000 UTC m=+252.297341588"
Jul 31 12:19:25.551566 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.551410    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:25.551566 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.551564    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:25.551801 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.551642    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:27.551619339 +0000 UTC m=+254.783484074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:25.753118 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:25.753027    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:25.753312 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.753243    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 12:19:25.753312 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:25.753308    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls podName:aa767e9c-0cfc-4f2c-93d5-15da50d670c8 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:27.753289197 +0000 UTC m=+254.985153942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jbl62" (UID: "aa767e9c-0cfc-4f2c-93d5-15da50d670c8") : secret "samples-operator-tls" not found
Jul 31 12:19:27.569247 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:27.568686    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:27.569247 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:27.568848    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:27.569247 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:27.568909    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:31.568889341 +0000 UTC m=+258.800754072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:27.770650 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:27.770617    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:27.770785 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:27.770739    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 12:19:27.770832 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:27.770817    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls podName:aa767e9c-0cfc-4f2c-93d5-15da50d670c8 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:31.770796174 +0000 UTC m=+259.002660910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jbl62" (UID: "aa767e9c-0cfc-4f2c-93d5-15da50d670c8") : secret "samples-operator-tls" not found
Jul 31 12:19:28.063445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.063374    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v" event={"ID":"8562adee-da47-4daf-8786-144e50002130","Type":"ContainerStarted","Data":"80eee0eb8de6d7a6aab6a363092daf67359a50eba882d5a95c97d63ae54a7294"}
Jul 31 12:19:28.064888 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.064863    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-vgntt" event={"ID":"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e","Type":"ContainerStarted","Data":"4b56aa73e75dd052dd7a7cca7de2c9a301dd065bfafa094d2f3b38ae6fac60d1"}
Jul 31 12:19:28.066393 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.066348    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb" event={"ID":"bdb35b15-fb85-4389-9a66-7ba41b50e853","Type":"ContainerStarted","Data":"bfa851630495784f6f0b03660c6acc515ef78c6f6ca651e2a9a7b6a42cdf43fe"}
Jul 31 12:19:28.067786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.067766    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4" event={"ID":"d7843221-f183-4ddd-b2d4-e4ded5d90b8d","Type":"ContainerStarted","Data":"57e72c293d69e2348f69152e2f6a4a313f56335596334f42d71544ac12bafd30"}
Jul 31 12:19:28.079617 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.079557    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v" podStartSLOduration=2.083478098 podStartE2EDuration="5.079539239s" totalImagesPullingTime="2.996061141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:28.078504205 +0000 UTC m=+255.310368981" watchObservedRunningTime="2026-07-31 12:19:28.079539239 +0000 UTC m=+255.311404044"
Jul 31 12:19:28.095092 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.095051    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-vgntt" podStartSLOduration=1.9930118129999999 podStartE2EDuration="5.095036697s" totalImagesPullingTime="3.102024884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:28.094372927 +0000 UTC m=+255.326237677" watchObservedRunningTime="2026-07-31 12:19:28.095036697 +0000 UTC m=+255.326901427"
Jul 31 12:19:28.108877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.108832    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb" podStartSLOduration=1.985237213 podStartE2EDuration="5.108817481s" totalImagesPullingTime="3.123580268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:28.108353658 +0000 UTC m=+255.340218411" watchObservedRunningTime="2026-07-31 12:19:28.108817481 +0000 UTC m=+255.340682216"
Jul 31 12:19:28.123025 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.122990    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-8rvn4" podStartSLOduration=2.115997884 podStartE2EDuration="5.122980932s" totalImagesPullingTime="3.006983048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:28.122716767 +0000 UTC m=+255.354581546" watchObservedRunningTime="2026-07-31 12:19:28.122980932 +0000 UTC m=+255.354845661"
Jul 31 12:19:28.482449 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.482403    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2w2jf"]
Jul 31 12:19:28.485657 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.485635    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.488132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.488106    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 31 12:19:28.488132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.488118    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qpgl9\""
Jul 31 12:19:28.488492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.488472    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 31 12:19:28.499458 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.499438    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2w2jf"]
Jul 31 12:19:28.577944 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.577916    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65406aef-7619-4ecb-8cc3-8e5783a0090f-data-volume\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.577944 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.577944    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.578372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.577965    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65406aef-7619-4ecb-8cc3-8e5783a0090f-crio-socket\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.578372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.578079    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.578372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.578108    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsn8v\" (UniqueName: \"kubernetes.io/projected/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-api-access-qsn8v\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.679625 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.679600    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65406aef-7619-4ecb-8cc3-8e5783a0090f-crio-socket\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680018 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.679993    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680079 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.680019    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65406aef-7619-4ecb-8cc3-8e5783a0090f-crio-socket\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680079 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.680041    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsn8v\" (UniqueName: \"kubernetes.io/projected/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-api-access-qsn8v\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680188 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.680119    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65406aef-7619-4ecb-8cc3-8e5783a0090f-data-volume\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680188 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:28.680126    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 12:19:28.680188 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.680149    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.680332 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:28.680212    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:29.180191701 +0000 UTC m=+256.412056433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : configmap "kube-rbac-proxy" not found
Jul 31 12:19:28.680332 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:28.680299    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:28.680437 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:28.680354    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:29.180338712 +0000 UTC m=+256.412203445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:28.684481 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.680734    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65406aef-7619-4ecb-8cc3-8e5783a0090f-data-volume\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:28.688798 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:28.688778    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsn8v\" (UniqueName: \"kubernetes.io/projected/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-api-access-qsn8v\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:29.095286 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.095253    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"]
Jul 31 12:19:29.097412 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.097391    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"
Jul 31 12:19:29.099671 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.099649    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 31 12:19:29.099791 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.099727    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 31 12:19:29.099791 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.099747    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mlmgp\""
Jul 31 12:19:29.106001 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.105977    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"]
Jul 31 12:19:29.184716 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.184679    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm57h\" (UniqueName: \"kubernetes.io/projected/76a9a985-ece6-482a-9aa7-824fc8c45a48-kube-api-access-rm57h\") pod \"migrator-68bc745d87-b2945\" (UID: \"76a9a985-ece6-482a-9aa7-824fc8c45a48\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"
Jul 31 12:19:29.184936 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.184901    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:29.185035 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:29.184954    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 12:19:29.185035 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:29.185008    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:30.184992977 +0000 UTC m=+257.416857709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : configmap "kube-rbac-proxy" not found
Jul 31 12:19:29.185315 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.185287    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:29.185417 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:29.185403    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:29.185455 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:29.185450    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:30.185435895 +0000 UTC m=+257.417300636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:29.286517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.286493    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm57h\" (UniqueName: \"kubernetes.io/projected/76a9a985-ece6-482a-9aa7-824fc8c45a48-kube-api-access-rm57h\") pod \"migrator-68bc745d87-b2945\" (UID: \"76a9a985-ece6-482a-9aa7-824fc8c45a48\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"
Jul 31 12:19:29.294762 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.294733    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm57h\" (UniqueName: \"kubernetes.io/projected/76a9a985-ece6-482a-9aa7-824fc8c45a48-kube-api-access-rm57h\") pod \"migrator-68bc745d87-b2945\" (UID: \"76a9a985-ece6-482a-9aa7-824fc8c45a48\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"
Jul 31 12:19:29.407533 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.407476    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"
Jul 31 12:19:29.522501 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:29.522466    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945"]
Jul 31 12:19:29.524984 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:29.524955    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a9a985_ece6_482a_9aa7_824fc8c45a48.slice/crio-c04909714950645a5bf5c07c2a33e58b0c60553bfee04afdd5efd3c791883a16 WatchSource:0}: Error finding container c04909714950645a5bf5c07c2a33e58b0c60553bfee04afdd5efd3c791883a16: Status 404 returned error can't find the container with id c04909714950645a5bf5c07c2a33e58b0c60553bfee04afdd5efd3c791883a16
Jul 31 12:19:30.075234 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:30.075195    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945" event={"ID":"76a9a985-ece6-482a-9aa7-824fc8c45a48","Type":"ContainerStarted","Data":"c04909714950645a5bf5c07c2a33e58b0c60553bfee04afdd5efd3c791883a16"}
Jul 31 12:19:30.192696 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:30.192663    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:30.192839 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:30.192747    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:30.192839 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:30.192805    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 12:19:30.192957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:30.192862    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:30.192957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:30.192880    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:32.192857556 +0000 UTC m=+259.424722303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : configmap "kube-rbac-proxy" not found
Jul 31 12:19:30.192957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:30.192910    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:32.192897657 +0000 UTC m=+259.424762388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:30.248670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:30.248648    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9lkp6_81c723fb-b45f-49ee-9a71-0f00ecb73e6b/dns-node-resolver/0.log"
Jul 31 12:19:31.079536 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.079466    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945" event={"ID":"76a9a985-ece6-482a-9aa7-824fc8c45a48","Type":"ContainerStarted","Data":"7bd4cfa84522805985734986fdb1ea4defb21b9d2992e69e9cc52a145cd7b156"}
Jul 31 12:19:31.079536 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.079509    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945" event={"ID":"76a9a985-ece6-482a-9aa7-824fc8c45a48","Type":"ContainerStarted","Data":"55cbcb148f28151e64e0cda1f4082c5e78fd2bd78537ff08a50455b6b5ee2f7c"}
Jul 31 12:19:31.094878 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.094838    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-b2945" podStartSLOduration=0.89937894 podStartE2EDuration="2.094823309s" totalImagesPullingTime="1.195444369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:31.0937835 +0000 UTC m=+258.325648251" watchObservedRunningTime="2026-07-31 12:19:31.094823309 +0000 UTC m=+258.326688059"
Jul 31 12:19:31.448290 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.448227    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dgnbk_50430d0c-78a9-4876-8dfe-9753de79bb9a/node-ca/0.log"
Jul 31 12:19:31.603141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.603112    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:31.603269 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:31.603252    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:31.603323 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:31.603312    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:39.603297547 +0000 UTC m=+266.835162276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:31.805551 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:31.805514    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:31.805687 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:31.805646    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 12:19:31.805730 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:31.805704    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls podName:aa767e9c-0cfc-4f2c-93d5-15da50d670c8 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:39.805690097 +0000 UTC m=+267.037554848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-jbl62" (UID: "aa767e9c-0cfc-4f2c-93d5-15da50d670c8") : secret "samples-operator-tls" not found
Jul 31 12:19:32.207565 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:32.207491    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:32.207565 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:32.207548    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:32.207957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:32.207605    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 12:19:32.207957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:32.207656    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:36.207640515 +0000 UTC m=+263.439505244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : configmap "kube-rbac-proxy" not found
Jul 31 12:19:32.207957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:32.207723    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:32.207957 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:32.207785    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:36.20776736 +0000 UTC m=+263.439632105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:32.250546 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:32.250525    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zlkhg_1107bc35-a8da-4171-a247-271408b87167/serve-healthcheck-canary/0.log"
Jul 31 12:19:36.236349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:36.236266    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:36.236349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:36.236325    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:36.236718 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:36.236374    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 12:19:36.236718 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:36.236437    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:44.236422211 +0000 UTC m=+271.468286940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : configmap "kube-rbac-proxy" not found
Jul 31 12:19:36.236718 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:36.236461    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:36.236718 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:36.236507    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:19:44.23649257 +0000 UTC m=+271.468357299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:39.661766 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:39.661736    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:39.662126 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:39.661870    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:39.662202 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:39.662131    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls podName:e553f719-737d-46b1-b8e7-b58473ee0a60 nodeName:}" failed. No retries permitted until 2026-07-31 12:19:55.662114098 +0000 UTC m=+282.893978827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-q7fd2" (UID: "e553f719-737d-46b1-b8e7-b58473ee0a60") : secret "cluster-monitoring-operator-tls" not found
Jul 31 12:19:39.862559 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:39.862528    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:39.864768 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:39.864739    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa767e9c-0cfc-4f2c-93d5-15da50d670c8-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-jbl62\" (UID: \"aa767e9c-0cfc-4f2c-93d5-15da50d670c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:39.975701 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:39.975676    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"
Jul 31 12:19:40.092116 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:40.092094    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62"]
Jul 31 12:19:41.116094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:41.116050    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62" event={"ID":"aa767e9c-0cfc-4f2c-93d5-15da50d670c8","Type":"ContainerStarted","Data":"478aaabbe0e364b2932f294f0bb5c3c06338a100ae70e85bb208c428803eaceb"}
Jul 31 12:19:42.121005 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:42.120923    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62" event={"ID":"aa767e9c-0cfc-4f2c-93d5-15da50d670c8","Type":"ContainerStarted","Data":"0608efa75e9c6b9a82cd0b3ea79f6e209cc51e63e0dea89f2a3f327e262edf5e"}
Jul 31 12:19:42.121005 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:42.120985    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62" event={"ID":"aa767e9c-0cfc-4f2c-93d5-15da50d670c8","Type":"ContainerStarted","Data":"84183e1a2de96ada3cec2944e07c01b02102ad831241d62e3259184b1df88794"}
Jul 31 12:19:42.137434 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:42.137388    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-jbl62" podStartSLOduration=16.535308662 podStartE2EDuration="18.137372861s" totalImagesPullingTime="1.602064199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:42.136894857 +0000 UTC m=+269.368759609" watchObservedRunningTime="2026-07-31 12:19:42.137372861 +0000 UTC m=+269.369237642"
Jul 31 12:19:44.299763 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:44.299729    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:44.300121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:44.299830    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:44.300121 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:44.299887    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:44.300121 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:44.299959    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls podName:65406aef-7619-4ecb-8cc3-8e5783a0090f nodeName:}" failed. No retries permitted until 2026-07-31 12:20:00.299938808 +0000 UTC m=+287.531803555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2w2jf" (UID: "65406aef-7619-4ecb-8cc3-8e5783a0090f") : secret "insights-runtime-extractor-tls" not found
Jul 31 12:19:44.300425 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:44.300408    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65406aef-7619-4ecb-8cc3-8e5783a0090f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:19:50.136310 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.136280    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-8m6n2"]
Jul 31 12:19:50.138935 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.138920    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:19:50.141108 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.141081    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 31 12:19:50.141304 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.141145    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j76c7\""
Jul 31 12:19:50.141449 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.141430    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 31 12:19:50.148612 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.148594    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-8m6n2"]
Jul 31 12:19:50.237181 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.237146    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plm2p\" (UniqueName: \"kubernetes.io/projected/f6203059-b243-4da7-9f75-a2464d2173e6-kube-api-access-plm2p\") pod \"downloads-77c974ffcf-8m6n2\" (UID: \"f6203059-b243-4da7-9f75-a2464d2173e6\") " pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:19:50.337632 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.337610    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plm2p\" (UniqueName: \"kubernetes.io/projected/f6203059-b243-4da7-9f75-a2464d2173e6-kube-api-access-plm2p\") pod \"downloads-77c974ffcf-8m6n2\" (UID: \"f6203059-b243-4da7-9f75-a2464d2173e6\") " pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:19:50.345495 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.345476    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plm2p\" (UniqueName: \"kubernetes.io/projected/f6203059-b243-4da7-9f75-a2464d2173e6-kube-api-access-plm2p\") pod \"downloads-77c974ffcf-8m6n2\" (UID: \"f6203059-b243-4da7-9f75-a2464d2173e6\") " pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:19:50.447943 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.447897    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:19:50.568610 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:50.568579    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-8m6n2"]
Jul 31 12:19:50.572922 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:50.572885    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6203059_b243_4da7_9f75_a2464d2173e6.slice/crio-8daa15cad8e5dee5756e8544b9ea00a372ab6a8b4fdc12e0693b776eccb04cb7 WatchSource:0}: Error finding container 8daa15cad8e5dee5756e8544b9ea00a372ab6a8b4fdc12e0693b776eccb04cb7: Status 404 returned error can't find the container with id 8daa15cad8e5dee5756e8544b9ea00a372ab6a8b4fdc12e0693b776eccb04cb7
Jul 31 12:19:51.148613 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:51.148577    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-8m6n2" event={"ID":"f6203059-b243-4da7-9f75-a2464d2173e6","Type":"ContainerStarted","Data":"8daa15cad8e5dee5756e8544b9ea00a372ab6a8b4fdc12e0693b776eccb04cb7"}
Jul 31 12:19:51.830351 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:51.830313    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wz22" podUID="de449ca7-7464-4981-825f-9c3fc3dead90"
Jul 31 12:19:51.830351 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:19:51.830324    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b"
Jul 31 12:19:52.152707 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:52.152626    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:19:52.153097 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:52.152626    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:19:55.479070 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.479037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:19:55.479552 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.479110    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:19:55.481651 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.481623    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de449ca7-7464-4981-825f-9c3fc3dead90-metrics-tls\") pod \"dns-default-8wz22\" (UID: \"de449ca7-7464-4981-825f-9c3fc3dead90\") " pod="openshift-dns/dns-default-8wz22"
Jul 31 12:19:55.481786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.481728    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"image-registry-77b9bc94b4-jcqjv\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") " pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:19:55.681428 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.681391    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:55.684136 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.684108    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e553f719-737d-46b1-b8e7-b58473ee0a60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-q7fd2\" (UID: \"e553f719-737d-46b1-b8e7-b58473ee0a60\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:55.693908 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.693887    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"
Jul 31 12:19:55.756679 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.756650    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tjczm\""
Jul 31 12:19:55.756679 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.756654    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ngjpv\""
Jul 31 12:19:55.764695 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.764625    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:19:55.764971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.764887    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:19:55.847365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.847284    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2"]
Jul 31 12:19:55.920663 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.920634    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wz22"]
Jul 31 12:19:55.924033 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:55.924010    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde449ca7_7464_4981_825f_9c3fc3dead90.slice/crio-578f16bb1feb758a1c0951209c13e2816d6608eff6e215d0b4fd9c3f552d26e5 WatchSource:0}: Error finding container 578f16bb1feb758a1c0951209c13e2816d6608eff6e215d0b4fd9c3f552d26e5: Status 404 returned error can't find the container with id 578f16bb1feb758a1c0951209c13e2816d6608eff6e215d0b4fd9c3f552d26e5
Jul 31 12:19:55.941232 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:55.941210    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:19:55.944427 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:19:55.944402    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38eb5134_0c8b_4b1d_868f_2eecaf7c827b.slice/crio-b1508456a0f44fb4057bb73650d95eef9dbddfc42e215be576c662c37ddfebd2 WatchSource:0}: Error finding container b1508456a0f44fb4057bb73650d95eef9dbddfc42e215be576c662c37ddfebd2: Status 404 returned error can't find the container with id b1508456a0f44fb4057bb73650d95eef9dbddfc42e215be576c662c37ddfebd2
Jul 31 12:19:56.168704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.168664    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wz22" event={"ID":"de449ca7-7464-4981-825f-9c3fc3dead90","Type":"ContainerStarted","Data":"578f16bb1feb758a1c0951209c13e2816d6608eff6e215d0b4fd9c3f552d26e5"}
Jul 31 12:19:56.170040 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.170004    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2" event={"ID":"e553f719-737d-46b1-b8e7-b58473ee0a60","Type":"ContainerStarted","Data":"911f0562e6fc51af242e5e486af32128d383fe06f3a734db9313b0845d644f08"}
Jul 31 12:19:56.171658 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.171630    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" event={"ID":"38eb5134-0c8b-4b1d-868f-2eecaf7c827b","Type":"ContainerStarted","Data":"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"}
Jul 31 12:19:56.171793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.171676    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" event={"ID":"38eb5134-0c8b-4b1d-868f-2eecaf7c827b","Type":"ContainerStarted","Data":"b1508456a0f44fb4057bb73650d95eef9dbddfc42e215be576c662c37ddfebd2"}
Jul 31 12:19:56.171793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.171781    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:19:56.192878 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:56.192783    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" podStartSLOduration=284.192765562 podStartE2EDuration="4m44.192765562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:56.19090217 +0000 UTC m=+283.422766939" watchObservedRunningTime="2026-07-31 12:19:56.192765562 +0000 UTC m=+283.424630315"
Jul 31 12:19:58.061157 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.061048    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:19:58.066014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.065982    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.069105 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069078    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 31 12:19:58.069237 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069111    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bk5bx\""
Jul 31 12:19:58.069237 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069117    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 31 12:19:58.069371 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069256    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 31 12:19:58.069429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069395    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 31 12:19:58.069489 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.069452    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 31 12:19:58.074015 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.073969    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:19:58.201554 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201524    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh97h\" (UniqueName: \"kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.201709 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201576    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.201709 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201612    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.201709 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201672    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.201854 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201727    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.201854 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.201778    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302377    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh97h\" (UniqueName: \"kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302459    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302491    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302524    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302556    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.302825 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.302594    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.303356 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.303333    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.303520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.303468    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.303520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.303504    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.305144 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.305122    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.305391 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.305372    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.311974 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.311914    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh97h\" (UniqueName: \"kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h\") pod \"console-66645f8d9b-j49br\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") " pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.378303 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.378277    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:19:58.544979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:58.544133    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:19:59.187201 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.187147    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66645f8d9b-j49br" event={"ID":"df88e04a-7a75-483b-a072-09655a3d7ff7","Type":"ContainerStarted","Data":"c377a596680a8bd96533bcdd0e856730bd4f51d38c0a85f38d35b50c43185d3e"}
Jul 31 12:19:59.189235 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.189206    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wz22" event={"ID":"de449ca7-7464-4981-825f-9c3fc3dead90","Type":"ContainerStarted","Data":"55b53024b5b473a8e02cd0595889a59fae6baaa6fcf3ae97de73918e7e215571"}
Jul 31 12:19:59.189365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.189257    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wz22" event={"ID":"de449ca7-7464-4981-825f-9c3fc3dead90","Type":"ContainerStarted","Data":"9ce068b96c5f1d450a8c1c4e4ca76c2eb472d6bc8ec172db2feeab37e1850dd5"}
Jul 31 12:19:59.189496 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.189467    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:19:59.191723 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.191674    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2" event={"ID":"e553f719-737d-46b1-b8e7-b58473ee0a60","Type":"ContainerStarted","Data":"34ccde03898f6fc5f492421a880fb3cbc440d6f988a4b77500cb78b04f5161d2"}
Jul 31 12:19:59.207918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.207804    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8wz22" podStartSLOduration=251.805895947 podStartE2EDuration="4m14.207785373s" totalImagesPullingTime="2.401889426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:59.207258845 +0000 UTC m=+286.439123594" watchObservedRunningTime="2026-07-31 12:19:59.207785373 +0000 UTC m=+286.439650126"
Jul 31 12:19:59.224549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:19:59.224500    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-q7fd2" podStartSLOduration=33.768347484 podStartE2EDuration="36.224483527s" totalImagesPullingTime="2.456136043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:19:59.223102132 +0000 UTC m=+286.454966886" watchObservedRunningTime="2026-07-31 12:19:59.224483527 +0000 UTC m=+286.456348279"
Jul 31 12:20:00.320547 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:00.320239    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:20:00.322849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:00.322821    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65406aef-7619-4ecb-8cc3-8e5783a0090f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2w2jf\" (UID: \"65406aef-7619-4ecb-8cc3-8e5783a0090f\") " pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:20:00.594791 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:00.594714    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2w2jf"
Jul 31 12:20:07.254185 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.254140    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-22nm4"]
Jul 31 12:20:07.257537 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.257516    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.259793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.259768    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 31 12:20:07.260509 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.260491    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 31 12:20:07.260608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.260538    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 31 12:20:07.260608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.260538    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 31 12:20:07.260608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.260595    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c5xjv\""
Jul 31 12:20:07.379804 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379777    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379822    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-textfile\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379857    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-sys\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379882    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-root\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379906    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-tls\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379961    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-wtmp\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.379981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.379982    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-metrics-client-ca\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.380270 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.380001    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.380270 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.380053    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjklt\" (UniqueName: \"kubernetes.io/projected/6664d00e-e027-4f7c-a1d2-fa4259507924-kube-api-access-gjklt\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481350 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481322    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-wtmp\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481494 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481368    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-metrics-client-ca\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481494 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481473    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481523    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjklt\" (UniqueName: \"kubernetes.io/projected/6664d00e-e027-4f7c-a1d2-fa4259507924-kube-api-access-gjklt\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481571    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481524    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-wtmp\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481713 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481644    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-textfile\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481713 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481695    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-sys\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481776 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481726    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-root\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481776 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481756    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-tls\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481881 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481867    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-root\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.481933 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481870    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6664d00e-e027-4f7c-a1d2-fa4259507924-sys\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.482016 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.481989    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-metrics-client-ca\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.482079 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.482061    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-textfile\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.482136 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.482117    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.484191 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.484155    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.484538 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.484492    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6664d00e-e027-4f7c-a1d2-fa4259507924-node-exporter-tls\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.489021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.489003    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjklt\" (UniqueName: \"kubernetes.io/projected/6664d00e-e027-4f7c-a1d2-fa4259507924-kube-api-access-gjklt\") pod \"node-exporter-22nm4\" (UID: \"6664d00e-e027-4f7c-a1d2-fa4259507924\") " pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.568831 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.568767    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22nm4"
Jul 31 12:20:07.643458 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.643426    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:07.646673 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.646649    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.654089 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.653909    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 31 12:20:07.658226 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.658203    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:07.784682 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784641    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.784825 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784690    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.784825 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784805    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.784931 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784842    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tfcw\" (UniqueName: \"kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.784931 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784900    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.785009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.784967    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.785056 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.785023    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.885868 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885801    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.885868 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885848    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885878    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885938    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885960    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tfcw\" (UniqueName: \"kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885980    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.885995    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886600 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.886567    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886806 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.886781    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886806 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.886796    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.886924 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.886839    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.888365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.888336    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.888450 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.888378    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.894389 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.894368    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tfcw\" (UniqueName: \"kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw\") pod \"console-666bbd74b6-rkcbv\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") " pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.957059 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:07.957037    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:07.968607 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:07.968578    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6664d00e_e027_4f7c_a1d2_fa4259507924.slice/crio-1846a06b57531b893e2cce415e66130ca128d64edeca922654737119c4e71a1f WatchSource:0}: Error finding container 1846a06b57531b893e2cce415e66130ca128d64edeca922654737119c4e71a1f: Status 404 returned error can't find the container with id 1846a06b57531b893e2cce415e66130ca128d64edeca922654737119c4e71a1f
Jul 31 12:20:08.104933 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.104911    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2w2jf"]
Jul 31 12:20:08.109812 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:08.109785    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65406aef_7619_4ecb_8cc3_8e5783a0090f.slice/crio-67cafccd35e0642d9f0660b6f42c3a4ed3ba59ae3bc3c63dcf8114329d7ed0fd WatchSource:0}: Error finding container 67cafccd35e0642d9f0660b6f42c3a4ed3ba59ae3bc3c63dcf8114329d7ed0fd: Status 404 returned error can't find the container with id 67cafccd35e0642d9f0660b6f42c3a4ed3ba59ae3bc3c63dcf8114329d7ed0fd
Jul 31 12:20:08.122533 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.122506    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:08.125617 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:08.125596    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7a0852_0d71_4335_8dce_70cabe334532.slice/crio-e9c15d94617acfc212021a2b1e4ad88e41e59c51cd04a24b82802efb3c409409 WatchSource:0}: Error finding container e9c15d94617acfc212021a2b1e4ad88e41e59c51cd04a24b82802efb3c409409: Status 404 returned error can't find the container with id e9c15d94617acfc212021a2b1e4ad88e41e59c51cd04a24b82802efb3c409409
Jul 31 12:20:08.228934 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.228894    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nm4" event={"ID":"6664d00e-e027-4f7c-a1d2-fa4259507924","Type":"ContainerStarted","Data":"1846a06b57531b893e2cce415e66130ca128d64edeca922654737119c4e71a1f"}
Jul 31 12:20:08.231492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.231465    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666bbd74b6-rkcbv" event={"ID":"9e7a0852-0d71-4335-8dce-70cabe334532","Type":"ContainerStarted","Data":"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"}
Jul 31 12:20:08.231618 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.231514    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666bbd74b6-rkcbv" event={"ID":"9e7a0852-0d71-4335-8dce-70cabe334532","Type":"ContainerStarted","Data":"e9c15d94617acfc212021a2b1e4ad88e41e59c51cd04a24b82802efb3c409409"}
Jul 31 12:20:08.233615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.233576    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2w2jf" event={"ID":"65406aef-7619-4ecb-8cc3-8e5783a0090f","Type":"ContainerStarted","Data":"6e8a0fda44316c22acad0e1639e01ebcf7121b473ab50d10134894d062fc16af"}
Jul 31 12:20:08.233706 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.233640    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2w2jf" event={"ID":"65406aef-7619-4ecb-8cc3-8e5783a0090f","Type":"ContainerStarted","Data":"67cafccd35e0642d9f0660b6f42c3a4ed3ba59ae3bc3c63dcf8114329d7ed0fd"}
Jul 31 12:20:08.235492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.235429    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66645f8d9b-j49br" event={"ID":"df88e04a-7a75-483b-a072-09655a3d7ff7","Type":"ContainerStarted","Data":"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"}
Jul 31 12:20:08.237340 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.237315    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-8m6n2" event={"ID":"f6203059-b243-4da7-9f75-a2464d2173e6","Type":"ContainerStarted","Data":"8c7bb76ff9b82cdfe1e2efdea0bd862d83e29baa226a07012d6ebc28552e8dc3"}
Jul 31 12:20:08.237718 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.237676    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:20:08.238886 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.238830    2532 patch_prober.go:28] interesting pod/downloads-77c974ffcf-8m6n2 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.18:8080/\": dial tcp 10.133.0.18:8080: connect: connection refused" start-of-body=
Jul 31 12:20:08.238993 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.238877    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-77c974ffcf-8m6n2" podUID="f6203059-b243-4da7-9f75-a2464d2173e6" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.18:8080/\": dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 31 12:20:08.271278 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.271223    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-8m6n2" podStartSLOduration=0.818956198 podStartE2EDuration="18.271204872s" totalImagesPullingTime="17.452248674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:08.270976383 +0000 UTC m=+295.502841134" watchObservedRunningTime="2026-07-31 12:20:08.271204872 +0000 UTC m=+295.503069623"
Jul 31 12:20:08.272629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.272588    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-666bbd74b6-rkcbv" podStartSLOduration=1.272574779 podStartE2EDuration="1.272574779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:08.252136711 +0000 UTC m=+295.484001463" watchObservedRunningTime="2026-07-31 12:20:08.272574779 +0000 UTC m=+295.504439530"
Jul 31 12:20:08.288305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.288261    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66645f8d9b-j49br" podStartSLOduration=0.871340556 podStartE2EDuration="10.2882445s" totalImagesPullingTime="9.416903944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:08.286660351 +0000 UTC m=+295.518525103" watchObservedRunningTime="2026-07-31 12:20:08.2882445 +0000 UTC m=+295.520109252"
Jul 31 12:20:08.307363 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.307338    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 12:20:08.312608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.312589    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.315280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315018    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 31 12:20:08.315280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315037    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 31 12:20:08.315280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315087    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 31 12:20:08.315280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315125    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 31 12:20:08.315280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315018    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 31 12:20:08.315913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315698    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 31 12:20:08.315913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315745    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 31 12:20:08.315913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315761    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 31 12:20:08.316102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.315996    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t6zmq\""
Jul 31 12:20:08.316102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.316015    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 31 12:20:08.327936 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.327911    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 12:20:08.341510 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:08.341470    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-59h74" podUID="f581f940-84e7-41b3-a723-29e3e33fceac"
Jul 31 12:20:08.382232 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.382205    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:20:08.382364 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.382258    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:20:08.383491 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.383461    2532 patch_prober.go:28] interesting pod/console-66645f8d9b-j49br container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused" start-of-body=
Jul 31 12:20:08.383597 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.383516    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-66645f8d9b-j49br" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerName="console" probeResult="failure" output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused"
Jul 31 12:20:08.390667 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390599    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390667 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390642    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdvd5\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-kube-api-access-gdvd5\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390685    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-config-volume\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390720    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390743    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390784    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390978 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390851    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390978 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390900    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-web-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390978 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390941    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-config-out\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.390978 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.390963    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.391133 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.391018    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.391133 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.391070    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.391133 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.391106    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.492587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.492089    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.492836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.492814    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-web-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.492961 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.492946    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-config-out\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493060 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493045    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493191 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493160    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493318 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493303    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493423    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493591 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493576    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493686 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493672    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdvd5\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-kube-api-access-gdvd5\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493799 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493784    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-config-volume\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.493910 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493896    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.494000 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.493988    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.494109 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.494095    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.494333 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:08.494316    2532 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 31 12:20:08.494486 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:08.494475    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls podName:54a39662-6818-4787-a665-ebeb60173157 nodeName:}" failed. No retries permitted until 2026-07-31 12:20:08.994454208 +0000 UTC m=+296.226318941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "54a39662-6818-4787-a665-ebeb60173157") : secret "alertmanager-main-tls" not found
Jul 31 12:20:08.495205 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.495144    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.496836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.496188    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.496836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.496482    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a39662-6818-4787-a665-ebeb60173157-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.497041 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.496942    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.497441 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.497116    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.498239 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.497981    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-config-volume\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.498239 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.498203    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-web-config\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.500024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.499995    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.500024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.500004    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.500163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.500042    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54a39662-6818-4787-a665-ebeb60173157-config-out\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.501564 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.501519    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.504302 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.504278    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdvd5\" (UniqueName: \"kubernetes.io/projected/54a39662-6818-4787-a665-ebeb60173157-kube-api-access-gdvd5\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:08.999395 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:08.999180    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:09.002424 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.002366    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54a39662-6818-4787-a665-ebeb60173157-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54a39662-6818-4787-a665-ebeb60173157\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:09.200221 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.199945    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8wz22"
Jul 31 12:20:09.223618 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.223571    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 12:20:09.255732 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.253414    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nm4" event={"ID":"6664d00e-e027-4f7c-a1d2-fa4259507924","Type":"ContainerStarted","Data":"ad8ec28b7cdab403789d21b977a3b3a23dd2865cef8fd67d8da225f7ff6a54cc"}
Jul 31 12:20:09.259659 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.259608    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-8m6n2"
Jul 31 12:20:09.462636 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:09.462402    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 12:20:09.467506 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:09.467471    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a39662_6818_4787_a665_ebeb60173157.slice/crio-2add9d93763326ce68437b8ff2abb8b02a5b2dab7ae314ff0a87129774e527c3 WatchSource:0}: Error finding container 2add9d93763326ce68437b8ff2abb8b02a5b2dab7ae314ff0a87129774e527c3: Status 404 returned error can't find the container with id 2add9d93763326ce68437b8ff2abb8b02a5b2dab7ae314ff0a87129774e527c3
Jul 31 12:20:10.231048 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.230864    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"]
Jul 31 12:20:10.255306 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.255251    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"]
Jul 31 12:20:10.255834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.255815    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.273548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.268963    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e6m2lmd4kp0am\""
Jul 31 12:20:10.273548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.269248    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 31 12:20:10.273548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.269504    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 31 12:20:10.273548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.269739    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-txk57\""
Jul 31 12:20:10.273548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.270748    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 31 12:20:10.276915 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.276583    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 31 12:20:10.276915 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.276618    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 31 12:20:10.281799 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.281766    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2w2jf" event={"ID":"65406aef-7619-4ecb-8cc3-8e5783a0090f","Type":"ContainerStarted","Data":"e61b9f741aa52435835d4b3da0e27e2a2a8489757a28c85defbb5049b99962bc"}
Jul 31 12:20:10.284213 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.284057    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"2add9d93763326ce68437b8ff2abb8b02a5b2dab7ae314ff0a87129774e527c3"}
Jul 31 12:20:10.286366 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.286330    2532 generic.go:356] "Generic (PLEG): container finished" podID="6664d00e-e027-4f7c-a1d2-fa4259507924" containerID="ad8ec28b7cdab403789d21b977a3b3a23dd2865cef8fd67d8da225f7ff6a54cc" exitCode=0
Jul 31 12:20:10.286523 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.286505    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nm4" event={"ID":"6664d00e-e027-4f7c-a1d2-fa4259507924","Type":"ContainerDied","Data":"ad8ec28b7cdab403789d21b977a3b3a23dd2865cef8fd67d8da225f7ff6a54cc"}
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313389    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzv8h\" (UniqueName: \"kubernetes.io/projected/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-kube-api-access-bzv8h\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313436    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313509    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313582    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313715    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-grpc-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313747    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313777    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.315121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.313818    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-metrics-client-ca\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415577    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzv8h\" (UniqueName: \"kubernetes.io/projected/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-kube-api-access-bzv8h\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415630    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415674    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415713    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415765    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-grpc-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415789    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415812    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.415877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.415841    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-metrics-client-ca\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.418659 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.416751    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-metrics-client-ca\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.423118 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.423090    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.423224 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.423194    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.426801 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.426330    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.426801 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.426607    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.427110 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.427048    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-grpc-tls\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.428875 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.428817    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.431559 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.431517    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzv8h\" (UniqueName: \"kubernetes.io/projected/5459e6e5-1a04-483a-ae95-51bfb7d7f23c-kube-api-access-bzv8h\") pod \"thanos-querier-6c49f57fc4-vrj8f\" (UID: \"5459e6e5-1a04-483a-ae95-51bfb7d7f23c\") " pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:10.583558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:10.583355    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:11.294220 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.294162    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nm4" event={"ID":"6664d00e-e027-4f7c-a1d2-fa4259507924","Type":"ContainerStarted","Data":"dd285936e7a45b83551c2eaf464d67229114abd27ed2147481500f5ebe6783f0"}
Jul 31 12:20:11.294411 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.294247    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nm4" event={"ID":"6664d00e-e027-4f7c-a1d2-fa4259507924","Type":"ContainerStarted","Data":"0c669291e65990a9ffa4b1956970ff06222dec238a0c5bdec3049d68b9d3f859"}
Jul 31 12:20:11.313446 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.313337    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-22nm4" podStartSLOduration=3.00265718 podStartE2EDuration="4.313319713s" totalImagesPullingTime="1.167200223s" totalInitContainerRuntime="143.46231ms" isStatefulPod=true podCreationTimestamp="2026-07-31 12:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:11.312010871 +0000 UTC m=+298.543875622" watchObservedRunningTime="2026-07-31 12:20:11.313319713 +0000 UTC m=+298.545184502"
Jul 31 12:20:11.421660 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.421613    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:20:11.426813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.426773    2532 patch_prober.go:28] interesting pod/image-registry-77b9bc94b4-jcqjv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 31 12:20:11.426937 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.426828    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 12:20:11.540837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.540789    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6df59b9d97-mmh8z"]
Jul 31 12:20:11.557341 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.557286    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6df59b9d97-mmh8z"]
Jul 31 12:20:11.557454 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.557393    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.559989 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.559961    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 31 12:20:11.560119 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.560039    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-95vgw\""
Jul 31 12:20:11.560119 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.560049    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 31 12:20:11.560119 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.560063    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 31 12:20:11.560119 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.560044    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-48unl53ugg58o\""
Jul 31 12:20:11.560386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.560316    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 31 12:20:11.625293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625267    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-metrics-server-audit-profiles\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625295    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-client-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625337    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-client-certs\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625476    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfzvq\" (UniqueName: \"kubernetes.io/projected/76882839-9656-4ba4-9f92-8c26a2157e9c-kube-api-access-qfzvq\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625514    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-tls\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625564    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.625721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.625610    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76882839-9656-4ba4-9f92-8c26a2157e9c-audit-log\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.726725 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726702    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfzvq\" (UniqueName: \"kubernetes.io/projected/76882839-9656-4ba4-9f92-8c26a2157e9c-kube-api-access-qfzvq\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.726873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726732    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-tls\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.726873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726766    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.726873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726804    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76882839-9656-4ba4-9f92-8c26a2157e9c-audit-log\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.726873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726862    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-metrics-server-audit-profiles\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.727116 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726884    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-client-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.727116 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.726918    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-client-certs\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.727548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.727503    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76882839-9656-4ba4-9f92-8c26a2157e9c-audit-log\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.727916 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.727842    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.728324 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.728301    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76882839-9656-4ba4-9f92-8c26a2157e9c-metrics-server-audit-profiles\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.729651 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.729619    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-client-certs\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.729741 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.729682    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-secret-metrics-server-tls\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.730281 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.730255    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76882839-9656-4ba4-9f92-8c26a2157e9c-client-ca-bundle\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.734853 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.734837    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfzvq\" (UniqueName: \"kubernetes.io/projected/76882839-9656-4ba4-9f92-8c26a2157e9c-kube-api-access-qfzvq\") pod \"metrics-server-6df59b9d97-mmh8z\" (UID: \"76882839-9656-4ba4-9f92-8c26a2157e9c\") " pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.868402 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.868366    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:11.951802 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:11.951717    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"]
Jul 31 12:20:11.960388 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:11.960221    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5459e6e5_1a04_483a_ae95_51bfb7d7f23c.slice/crio-86c32fecbd13347460e89ca107ab84c5d2b52c49d18d6b830bab40a8e1297d58 WatchSource:0}: Error finding container 86c32fecbd13347460e89ca107ab84c5d2b52c49d18d6b830bab40a8e1297d58: Status 404 returned error can't find the container with id 86c32fecbd13347460e89ca107ab84c5d2b52c49d18d6b830bab40a8e1297d58
Jul 31 12:20:12.021612 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.021582    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"]
Jul 31 12:20:12.039499 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.039471    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"]
Jul 31 12:20:12.039638 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.039615    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.041716 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.041693    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 31 12:20:12.041716 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.041707    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l258q\""
Jul 31 12:20:12.050577 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.050551    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6df59b9d97-mmh8z"]
Jul 31 12:20:12.054351 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:12.054327    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76882839_9656_4ba4_9f92_8c26a2157e9c.slice/crio-05f36c2907ee150e237d04dd32853574ca221589e30e8aaf9627c18a5ab3d4e5 WatchSource:0}: Error finding container 05f36c2907ee150e237d04dd32853574ca221589e30e8aaf9627c18a5ab3d4e5: Status 404 returned error can't find the container with id 05f36c2907ee150e237d04dd32853574ca221589e30e8aaf9627c18a5ab3d4e5
Jul 31 12:20:12.130928 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.130862    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlz7\" (UniqueName: \"kubernetes.io/projected/050fd95a-c892-42e3-b840-8d852f94864d-kube-api-access-kqlz7\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.130928 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.130899    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/050fd95a-c892-42e3-b840-8d852f94864d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.231585 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.231554    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlz7\" (UniqueName: \"kubernetes.io/projected/050fd95a-c892-42e3-b840-8d852f94864d-kube-api-access-kqlz7\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.231770 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.231601    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/050fd95a-c892-42e3-b840-8d852f94864d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.234135 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.234112    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/050fd95a-c892-42e3-b840-8d852f94864d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.239551 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.239524    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlz7\" (UniqueName: \"kubernetes.io/projected/050fd95a-c892-42e3-b840-8d852f94864d-kube-api-access-kqlz7\") pod \"monitoring-plugin-6dffbbd67-8dt42\" (UID: \"050fd95a-c892-42e3-b840-8d852f94864d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.299780 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.299751    2532 generic.go:356] "Generic (PLEG): container finished" podID="54a39662-6818-4787-a665-ebeb60173157" containerID="549997e248e5e4d3d93868c7d76f04601851a62817c2efb7067d534de3ea6159" exitCode=0
Jul 31 12:20:12.299891 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.299848    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerDied","Data":"549997e248e5e4d3d93868c7d76f04601851a62817c2efb7067d534de3ea6159"}
Jul 31 12:20:12.301319 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.301297    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"86c32fecbd13347460e89ca107ab84c5d2b52c49d18d6b830bab40a8e1297d58"}
Jul 31 12:20:12.302749 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.302726    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z" event={"ID":"76882839-9656-4ba4-9f92-8c26a2157e9c","Type":"ContainerStarted","Data":"05f36c2907ee150e237d04dd32853574ca221589e30e8aaf9627c18a5ab3d4e5"}
Jul 31 12:20:12.305451 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.305428    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2w2jf" event={"ID":"65406aef-7619-4ecb-8cc3-8e5783a0090f","Type":"ContainerStarted","Data":"b5ca0ff9871c1ff05243581c1bdadc37e418ffba28162b9f0035137b8bcd76b1"}
Jul 31 12:20:12.350734 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.350688    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2w2jf" podStartSLOduration=40.813436765 podStartE2EDuration="44.350675447s" totalImagesPullingTime="3.537238682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 12:19:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:12.349445932 +0000 UTC m=+299.581310698" watchObservedRunningTime="2026-07-31 12:20:12.350675447 +0000 UTC m=+299.582540197"
Jul 31 12:20:12.350928 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.350908    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:12.516489 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:12.516388    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"]
Jul 31 12:20:12.520340 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:12.520250    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050fd95a_c892_42e3_b840_8d852f94864d.slice/crio-23cdd9aef64bf7506bb025a507d010c525d12eb98f7d404dee4d124deefe0038 WatchSource:0}: Error finding container 23cdd9aef64bf7506bb025a507d010c525d12eb98f7d404dee4d124deefe0038: Status 404 returned error can't find the container with id 23cdd9aef64bf7506bb025a507d010c525d12eb98f7d404dee4d124deefe0038
Jul 31 12:20:13.267014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.266909    2532 kubelet.go:1648] "Image garbage collection succeeded"
Jul 31 12:20:13.314434 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.314228    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42" event={"ID":"050fd95a-c892-42e3-b840-8d852f94864d","Type":"ContainerStarted","Data":"23cdd9aef64bf7506bb025a507d010c525d12eb98f7d404dee4d124deefe0038"}
Jul 31 12:20:13.689278 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.689065    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:20:13.718862 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.718574    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:20:13.751028 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.750999    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:20:13.751213 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.751111    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848344 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848205    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848344 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848310    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppc28\" (UniqueName: \"kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848387    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848415    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848441    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848475    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.848580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.848507    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949431 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949290    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppc28\" (UniqueName: \"kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949431 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949375    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949431 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949407    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949435    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949467    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949646    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.949813 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.949734    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.950486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.950424    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.950486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.950459    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.951337 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.951294    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.951598 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.951580    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.952492 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.952451    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.953279 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.953253    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:13.958753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:13.958728    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppc28\" (UniqueName: \"kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28\") pod \"console-6c7fd98574-bdstw\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") " pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:14.065472 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:14.065401    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:16.881201 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:16.880953    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:20:16.886859 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:20:16.886802    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9910e09_8388_432d_9921_835006af57ae.slice/crio-cdfa50cc45a61136f07b5142753f2a77f2de0b2a0703e102b800e5cea77dbd7f WatchSource:0}: Error finding container cdfa50cc45a61136f07b5142753f2a77f2de0b2a0703e102b800e5cea77dbd7f: Status 404 returned error can't find the container with id cdfa50cc45a61136f07b5142753f2a77f2de0b2a0703e102b800e5cea77dbd7f
Jul 31 12:20:17.156825 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.156454    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 12:20:17.350687 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.350623    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"82fa782627021573acf5eb5cae0eb175bd1bcad08bc8654b2acd94720de9d962"}
Jul 31 12:20:17.350807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.350706    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"e8005c64af79a8f55a1e40cffa2c650daaf2af1b19ae8eb1572c32ec2c0185d8"}
Jul 31 12:20:17.350807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.350743    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"2821b99c7f83264e4b5ea1cdafcb2b79970c787b656e391792d097dbdad3a78b"}
Jul 31 12:20:17.350807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.350780    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"4fb4b0e0296b69ae91bb67b47543ec02e71d05831e97349453d609982f56250b"}
Jul 31 12:20:17.352891 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.352857    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"a3c83918065a1197b7a442b26925d33aef010451e4408aa310379f7c6cc9303b"}
Jul 31 12:20:17.352979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.352935    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"4a59afe51dd929170c55ba1698b4e81fa68363074fa7aaa66847b8cecb2d8608"}
Jul 31 12:20:17.353026 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.352985    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"26d709caade5985c5153be9c2f8d5543f31f969d1a8d7a6e778b1b0c84a7714b"}
Jul 31 12:20:17.354648 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.354623    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z" event={"ID":"76882839-9656-4ba4-9f92-8c26a2157e9c","Type":"ContainerStarted","Data":"ec572b83b20149d50988b03c4a4a00c7d2d69c040afe5a5cfd0c1cae4d374f5d"}
Jul 31 12:20:17.356361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.356324    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fd98574-bdstw" event={"ID":"d9910e09-8388-432d-9921-835006af57ae","Type":"ContainerStarted","Data":"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"}
Jul 31 12:20:17.356454 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.356365    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fd98574-bdstw" event={"ID":"d9910e09-8388-432d-9921-835006af57ae","Type":"ContainerStarted","Data":"cdfa50cc45a61136f07b5142753f2a77f2de0b2a0703e102b800e5cea77dbd7f"}
Jul 31 12:20:17.357772 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.357748    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42" event={"ID":"050fd95a-c892-42e3-b840-8d852f94864d","Type":"ContainerStarted","Data":"7c8721241818e65cc3be3dc202488a394cac17a0a02b3f349430cb278307e24b"}
Jul 31 12:20:17.357959 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.357944    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:17.362665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.362646    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42"
Jul 31 12:20:17.372227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.372191    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z" podStartSLOduration=1.7097902459999998 podStartE2EDuration="6.372160185s" totalImagesPullingTime="4.662369939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:17.371698427 +0000 UTC m=+304.603563189" watchObservedRunningTime="2026-07-31 12:20:17.372160185 +0000 UTC m=+304.604025003"
Jul 31 12:20:17.386146 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.386107    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8dt42" podStartSLOduration=1.1900743010000001 podStartE2EDuration="5.38609513s" totalImagesPullingTime="4.196020829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:17.385366515 +0000 UTC m=+304.617231266" watchObservedRunningTime="2026-07-31 12:20:17.38609513 +0000 UTC m=+304.617959921"
Jul 31 12:20:17.403467 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.403425    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c7fd98574-bdstw" podStartSLOduration=4.403410213 podStartE2EDuration="4.403410213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:17.402106548 +0000 UTC m=+304.633971319" watchObservedRunningTime="2026-07-31 12:20:17.403410213 +0000 UTC m=+304.635274965"
Jul 31 12:20:17.958007 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.957972    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:17.958580 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.958080    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:17.963482 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:17.963457    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:18.366743 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:18.366707    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"9341d936ed14ac4a60236460575230efd49bacd3b9d9882f65dcfb88b94c2466"}
Jul 31 12:20:18.370499 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:18.370479    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:19.373790 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.373761    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54a39662-6818-4787-a665-ebeb60173157","Type":"ContainerStarted","Data":"ae8f5cbd05cd4efee3362ab373744b41966b42ae3deb48ed01c58f3c7410f6a5"}
Jul 31 12:20:19.376569 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.376537    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"2200c022c6f1d7139cb8c1cf5688189d7ce67398ae7d76d3f0a29af838dc7d07"}
Jul 31 12:20:19.376684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.376611    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"b4ffc6bfd77d9eb9fba8893cb202e9157126a4d9bb5f9c764db5204bde59701a"}
Jul 31 12:20:19.376857 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.376841    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:19.376934 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.376893    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" event={"ID":"5459e6e5-1a04-483a-ae95-51bfb7d7f23c","Type":"ContainerStarted","Data":"0288f1af04a49c7728dd4ae035eeb61a018b85c87094d646d256d03bf40b892e"}
Jul 31 12:20:19.399303 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.399264    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.406378188 podStartE2EDuration="11.399250897s" totalImagesPullingTime="7.985400987s" totalInitContainerRuntime="7.471722ms" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:19.397436159 +0000 UTC m=+306.629300910" watchObservedRunningTime="2026-07-31 12:20:19.399250897 +0000 UTC m=+306.631115676"
Jul 31 12:20:19.417748 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:19.417710    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f" podStartSLOduration=3.167560625 podStartE2EDuration="9.417698259s" totalImagesPullingTime="6.250137634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:20:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:20:19.416664891 +0000 UTC m=+306.648529667" watchObservedRunningTime="2026-07-31 12:20:19.417698259 +0000 UTC m=+306.649563031"
Jul 31 12:20:21.425952 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:21.425926    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:20:23.341659 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:23.341632    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:20:24.066488 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:24.066457    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:24.066648 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:24.066528    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:24.071060 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:24.071040    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:24.400291 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:24.400224    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:20:24.443279 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:24.443249    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:25.386050 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:25.386025    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c49f57fc4-vrj8f"
Jul 31 12:20:31.868983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:31.868955    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:31.869587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:31.869008    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:36.442166 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.442127    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerName="registry" containerID="cri-o://dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8" gracePeriod=30
Jul 31 12:20:36.673110 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.673089    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:20:36.741826 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.741802    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.741949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.741843    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.741949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.741878    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.741949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.741905    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.742093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.741948    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.742093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.742002    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.742093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.742044    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6bk2\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.742093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.742070    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") pod \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\" (UID: \"38eb5134-0c8b-4b1d-868f-2eecaf7c827b\") "
Jul 31 12:20:36.742683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.742596    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:36.742683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.742646    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:36.744461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.744429    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2" (OuterVolumeSpecName: "kube-api-access-s6bk2") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "kube-api-access-s6bk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:20:36.744559 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.744512    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:20:36.744608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.744581    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:36.744673 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.744658    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:36.744720 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.744665    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:20:36.753498 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.753478    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "38eb5134-0c8b-4b1d-868f-2eecaf7c827b" (UID: "38eb5134-0c8b-4b1d-868f-2eecaf7c827b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:20:36.842981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.842958    2532 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-bound-sa-token\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.842981 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.842977    2532 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-certificates\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.842988    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-trusted-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.842997    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6bk2\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-kube-api-access-s6bk2\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.843004    2532 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-registry-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.843012    2532 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-image-registry-private-configuration\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.843021    2532 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-ca-trust-extracted\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:36.843102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:36.843029    2532 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38eb5134-0c8b-4b1d-868f-2eecaf7c827b-installation-pull-secrets\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:37.454278 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.454248    2532 generic.go:356] "Generic (PLEG): container finished" podID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerID="dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8" exitCode=0
Jul 31 12:20:37.454708 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.454288    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" event={"ID":"38eb5134-0c8b-4b1d-868f-2eecaf7c827b","Type":"ContainerDied","Data":"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"}
Jul 31 12:20:37.454708 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.454296    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv"
Jul 31 12:20:37.454708 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.454323    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b9bc94b4-jcqjv" event={"ID":"38eb5134-0c8b-4b1d-868f-2eecaf7c827b","Type":"ContainerDied","Data":"b1508456a0f44fb4057bb73650d95eef9dbddfc42e215be576c662c37ddfebd2"}
Jul 31 12:20:37.454708 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.454352    2532 scope.go:117] "RemoveContainer" containerID="dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"
Jul 31 12:20:37.464293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.464275    2532 scope.go:117] "RemoveContainer" containerID="dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"
Jul 31 12:20:37.464554 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:37.464534    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8\": container with ID starting with dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8 not found: ID does not exist" containerID="dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"
Jul 31 12:20:37.464596 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.464562    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8"} err="failed to get container status \"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8\": rpc error: code = NotFound desc = could not find container \"dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8\": container with ID starting with dc8d4aa6ecca4ca7083e8a3d2b0775d531cc85ecf453ce7e60f44e96fcc097d8 not found: ID does not exist"
Jul 31 12:20:37.472329 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.472309    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:20:37.475459 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:37.475441    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77b9bc94b4-jcqjv"]
Jul 31 12:20:38.459349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:38.459324    2532 generic.go:356] "Generic (PLEG): container finished" podID="8562adee-da47-4daf-8786-144e50002130" containerID="80eee0eb8de6d7a6aab6a363092daf67359a50eba882d5a95c97d63ae54a7294" exitCode=0
Jul 31 12:20:38.459663 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:38.459395    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v" event={"ID":"8562adee-da47-4daf-8786-144e50002130","Type":"ContainerDied","Data":"80eee0eb8de6d7a6aab6a363092daf67359a50eba882d5a95c97d63ae54a7294"}
Jul 31 12:20:38.459757 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:38.459671    2532 scope.go:117] "RemoveContainer" containerID="80eee0eb8de6d7a6aab6a363092daf67359a50eba882d5a95c97d63ae54a7294"
Jul 31 12:20:38.723422 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:38.723393    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66645f8d9b-j49br" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerName="console" containerID="cri-o://62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee" gracePeriod=15
Jul 31 12:20:39.011085 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.011059    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:20:39.059697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059667    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh97h\" (UniqueName: \"kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.059829 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059709    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.059829 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059738    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.059829 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059782    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.059829 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059813    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.060030 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.059864    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config\") pod \"df88e04a-7a75-483b-a072-09655a3d7ff7\" (UID: \"df88e04a-7a75-483b-a072-09655a3d7ff7\") "
Jul 31 12:20:39.060184 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.060141    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:39.060317 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.060231    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca" (OuterVolumeSpecName: "service-ca") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:39.060376 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.060344    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config" (OuterVolumeSpecName: "console-config") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:39.061957 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.061933    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:39.062119 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.062101    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:39.062217 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.062125    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h" (OuterVolumeSpecName: "kube-api-access-rh97h") pod "df88e04a-7a75-483b-a072-09655a3d7ff7" (UID: "df88e04a-7a75-483b-a072-09655a3d7ff7"). InnerVolumeSpecName "kube-api-access-rh97h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:20:39.160942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160917    2532 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.160942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160938    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh97h\" (UniqueName: \"kubernetes.io/projected/df88e04a-7a75-483b-a072-09655a3d7ff7-kube-api-access-rh97h\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.161094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160949    2532 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.161094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160958    2532 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.161094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160966    2532 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df88e04a-7a75-483b-a072-09655a3d7ff7-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.161094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.160974    2532 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df88e04a-7a75-483b-a072-09655a3d7ff7-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:39.344966 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.344885    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" path="/var/lib/kubelet/pods/38eb5134-0c8b-4b1d-868f-2eecaf7c827b/volumes"
Jul 31 12:20:39.465656 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.465630    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-sbf7v" event={"ID":"8562adee-da47-4daf-8786-144e50002130","Type":"ContainerStarted","Data":"2ad82affc311d71d2c13f4e8876cb16de0ddb8be202778960395f30400332296"}
Jul 31 12:20:39.466873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.466850    2532 generic.go:356] "Generic (PLEG): container finished" podID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerID="62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee" exitCode=0
Jul 31 12:20:39.466977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.466886    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66645f8d9b-j49br"
Jul 31 12:20:39.466977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.466936    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66645f8d9b-j49br" event={"ID":"df88e04a-7a75-483b-a072-09655a3d7ff7","Type":"ContainerDied","Data":"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"}
Jul 31 12:20:39.467086 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.467000    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66645f8d9b-j49br" event={"ID":"df88e04a-7a75-483b-a072-09655a3d7ff7","Type":"ContainerDied","Data":"c377a596680a8bd96533bcdd0e856730bd4f51d38c0a85f38d35b50c43185d3e"}
Jul 31 12:20:39.467086 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.467033    2532 scope.go:117] "RemoveContainer" containerID="62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"
Jul 31 12:20:39.486791 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.486763    2532 scope.go:117] "RemoveContainer" containerID="62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"
Jul 31 12:20:39.487057 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:39.487037    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee\": container with ID starting with 62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee not found: ID does not exist" containerID="62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"
Jul 31 12:20:39.487146 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.487064    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee"} err="failed to get container status \"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee\": rpc error: code = NotFound desc = could not find container \"62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee\": container with ID starting with 62d1e20415cdd0b1024c533f7338e81905379eab20f789446d1037465bd343ee not found: ID does not exist"
Jul 31 12:20:39.494381 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.494360    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:20:39.500008 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:39.499984    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66645f8d9b-j49br"]
Jul 31 12:20:41.344865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:41.344822    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" path="/var/lib/kubelet/pods/df88e04a-7a75-483b-a072-09655a3d7ff7/volumes"
Jul 31 12:20:43.485930 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:43.485895    2532 generic.go:356] "Generic (PLEG): container finished" podID="8bd46abd-03c1-4f7d-88d0-8957be2aeb8e" containerID="4b56aa73e75dd052dd7a7cca7de2c9a301dd065bfafa094d2f3b38ae6fac60d1" exitCode=0
Jul 31 12:20:43.486304 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:43.485964    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-vgntt" event={"ID":"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e","Type":"ContainerDied","Data":"4b56aa73e75dd052dd7a7cca7de2c9a301dd065bfafa094d2f3b38ae6fac60d1"}
Jul 31 12:20:43.486304 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:43.486253    2532 scope.go:117] "RemoveContainer" containerID="4b56aa73e75dd052dd7a7cca7de2c9a301dd065bfafa094d2f3b38ae6fac60d1"
Jul 31 12:20:44.491578 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:44.491546    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-vgntt" event={"ID":"8bd46abd-03c1-4f7d-88d0-8957be2aeb8e","Type":"ContainerStarted","Data":"68fe6da8b855663e6386dc41fc863fe2a83e72bbd6a0a5bcda34e3815bf23417"}
Jul 31 12:20:49.461783 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.461742    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-666bbd74b6-rkcbv" podUID="9e7a0852-0d71-4335-8dce-70cabe334532" containerName="console" containerID="cri-o://c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9" gracePeriod=15
Jul 31 12:20:49.718111 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.718063    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:49.844833 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.844811    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.844949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.844849    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.844949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.844871    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.844949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.844893    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.845098 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845018    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.845098 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845090    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.845217 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845146    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tfcw\" (UniqueName: \"kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw\") pod \"9e7a0852-0d71-4335-8dce-70cabe334532\" (UID: \"9e7a0852-0d71-4335-8dce-70cabe334532\") "
Jul 31 12:20:49.845323 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845298    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:49.845376 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845308    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config" (OuterVolumeSpecName: "console-config") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:49.845376 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845337    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca" (OuterVolumeSpecName: "service-ca") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:49.845453 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845424    2532 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.845453 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845438    2532 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.845453 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845447    2532 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.845688 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.845666    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:20:49.847083 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.847055    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw" (OuterVolumeSpecName: "kube-api-access-2tfcw") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "kube-api-access-2tfcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:20:49.847410 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.847382    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:49.847501 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.847410    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9e7a0852-0d71-4335-8dce-70cabe334532" (UID: "9e7a0852-0d71-4335-8dce-70cabe334532"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:20:49.945891 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.945868    2532 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.945891 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.945888    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7a0852-0d71-4335-8dce-70cabe334532-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.946024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.945897    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tfcw\" (UniqueName: \"kubernetes.io/projected/9e7a0852-0d71-4335-8dce-70cabe334532-kube-api-access-2tfcw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:49.946024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:49.945907    2532 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7a0852-0d71-4335-8dce-70cabe334532-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:20:50.514850 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.514823    2532 generic.go:356] "Generic (PLEG): container finished" podID="9e7a0852-0d71-4335-8dce-70cabe334532" containerID="c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9" exitCode=0
Jul 31 12:20:50.515227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.514863    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666bbd74b6-rkcbv"
Jul 31 12:20:50.515227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.514910    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666bbd74b6-rkcbv" event={"ID":"9e7a0852-0d71-4335-8dce-70cabe334532","Type":"ContainerDied","Data":"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"}
Jul 31 12:20:50.515227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.514950    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666bbd74b6-rkcbv" event={"ID":"9e7a0852-0d71-4335-8dce-70cabe334532","Type":"ContainerDied","Data":"e9c15d94617acfc212021a2b1e4ad88e41e59c51cd04a24b82802efb3c409409"}
Jul 31 12:20:50.515227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.514970    2532 scope.go:117] "RemoveContainer" containerID="c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"
Jul 31 12:20:50.524068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.524051    2532 scope.go:117] "RemoveContainer" containerID="c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"
Jul 31 12:20:50.524318 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:20:50.524300    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9\": container with ID starting with c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9 not found: ID does not exist" containerID="c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"
Jul 31 12:20:50.524388 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.524324    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9"} err="failed to get container status \"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9\": rpc error: code = NotFound desc = could not find container \"c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9\": container with ID starting with c93f94bf08972f22679e6adfeae9838541f81685a65c86440dfa608bec942ef9 not found: ID does not exist"
Jul 31 12:20:50.535672 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.535635    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:50.538616 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:50.538598    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-666bbd74b6-rkcbv"]
Jul 31 12:20:51.344712 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:51.344681    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7a0852-0d71-4335-8dce-70cabe334532" path="/var/lib/kubelet/pods/9e7a0852-0d71-4335-8dce-70cabe334532/volumes"
Jul 31 12:20:51.874540 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:51.874513    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:51.878212 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:51.878190    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6df59b9d97-mmh8z"
Jul 31 12:20:53.533385 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:53.533353    2532 generic.go:356] "Generic (PLEG): container finished" podID="bdb35b15-fb85-4389-9a66-7ba41b50e853" containerID="bfa851630495784f6f0b03660c6acc515ef78c6f6ca651e2a9a7b6a42cdf43fe" exitCode=0
Jul 31 12:20:53.533726 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:53.533424    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb" event={"ID":"bdb35b15-fb85-4389-9a66-7ba41b50e853","Type":"ContainerDied","Data":"bfa851630495784f6f0b03660c6acc515ef78c6f6ca651e2a9a7b6a42cdf43fe"}
Jul 31 12:20:53.533726 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:53.533718    2532 scope.go:117] "RemoveContainer" containerID="bfa851630495784f6f0b03660c6acc515ef78c6f6ca651e2a9a7b6a42cdf43fe"
Jul 31 12:20:54.540896 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:20:54.540864    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-58hvb" event={"ID":"bdb35b15-fb85-4389-9a66-7ba41b50e853","Type":"ContainerStarted","Data":"4b9f2021d71a10a2df15de5f771ca75285f89a6089e80acb51642c0d96196bba"}
Jul 31 12:21:27.126103 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.126061    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:21:27.128227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.128209    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f581f940-84e7-41b3-a723-29e3e33fceac-metrics-certs\") pod \"network-metrics-daemon-59h74\" (UID: \"f581f940-84e7-41b3-a723-29e3e33fceac\") " pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:21:27.244976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.244952    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qqfkp\""
Jul 31 12:21:27.253520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.253505    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59h74"
Jul 31 12:21:27.378697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.378674    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59h74"]
Jul 31 12:21:27.381165 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:21:27.381138    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf581f940_84e7_41b3_a723_29e3e33fceac.slice/crio-92f7ca437ff5a9708e26106e2d6cae1d1142e1dc267433971211ef398c0fdf46 WatchSource:0}: Error finding container 92f7ca437ff5a9708e26106e2d6cae1d1142e1dc267433971211ef398c0fdf46: Status 404 returned error can't find the container with id 92f7ca437ff5a9708e26106e2d6cae1d1142e1dc267433971211ef398c0fdf46
Jul 31 12:21:27.678956 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:27.678889    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59h74" event={"ID":"f581f940-84e7-41b3-a723-29e3e33fceac","Type":"ContainerStarted","Data":"92f7ca437ff5a9708e26106e2d6cae1d1142e1dc267433971211ef398c0fdf46"}
Jul 31 12:21:28.684819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:28.684739    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59h74" event={"ID":"f581f940-84e7-41b3-a723-29e3e33fceac","Type":"ContainerStarted","Data":"1f233cb851f2a3dfc422f4ec439d7dc8fc187259dfa987d98da6f98877532ede"}
Jul 31 12:21:28.684819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:28.684798    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59h74" event={"ID":"f581f940-84e7-41b3-a723-29e3e33fceac","Type":"ContainerStarted","Data":"5c779d5702caaf38cfe1fa5114a84a89a20091068ea78a0f7a08d0407b1f555b"}
Jul 31 12:21:28.702666 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:28.702616    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-59h74" podStartSLOduration=374.869540529 podStartE2EDuration="6m15.702600709s" totalImagesPullingTime="833.06018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:21:28.70041326 +0000 UTC m=+375.932278023" watchObservedRunningTime="2026-07-31 12:21:28.702600709 +0000 UTC m=+375.934465459"
Jul 31 12:21:31.529657 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529625    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b658575dc-qvnpz"]
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529927    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerName="registry"
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529938    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerName="registry"
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529946    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7a0852-0d71-4335-8dce-70cabe334532" containerName="console"
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529951    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7a0852-0d71-4335-8dce-70cabe334532" containerName="console"
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529963    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerName="console"
Jul 31 12:21:31.530021 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.529969    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerName="console"
Jul 31 12:21:31.530244 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.530025    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e7a0852-0d71-4335-8dce-70cabe334532" containerName="console"
Jul 31 12:21:31.530244 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.530034    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="38eb5134-0c8b-4b1d-868f-2eecaf7c827b" containerName="registry"
Jul 31 12:21:31.530244 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.530039    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="df88e04a-7a75-483b-a072-09655a3d7ff7" containerName="console"
Jul 31 12:21:31.532267 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.532250    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.534974 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.534941    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 31 12:21:31.535113 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.535066    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 31 12:21:31.535113 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.535083    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 31 12:21:31.535281 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.535241    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 31 12:21:31.535405 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.535383    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pw2gs\""
Jul 31 12:21:31.535528 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.535511    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 31 12:21:31.540525 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.540508    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 31 12:21:31.544524 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.544496    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b658575dc-qvnpz"]
Jul 31 12:21:31.558753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558734    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.558856 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558774    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.558856 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558808    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-serving-certs-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.558856 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558840    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-metrics-client-ca\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.559009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558877    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-federate-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.559009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558907    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpwtt\" (UniqueName: \"kubernetes.io/projected/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-kube-api-access-xpwtt\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.559009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558938    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.559009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.558983    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659572    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659603    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-serving-certs-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659621    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-metrics-client-ca\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659643    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-federate-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659661    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpwtt\" (UniqueName: \"kubernetes.io/projected/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-kube-api-access-xpwtt\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659691 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659686    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659931 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659712    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.659931 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.659766    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.660403 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.660384    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-serving-certs-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.660514 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.660491    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.660671 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.660652    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-metrics-client-ca\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.662155 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.662128    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-telemeter-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.662255 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.662162    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.662464 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.662447    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-secret-telemeter-client\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.662617 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.662589    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-federate-client-tls\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.670746 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.670724    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpwtt\" (UniqueName: \"kubernetes.io/projected/b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a-kube-api-access-xpwtt\") pod \"telemeter-client-6b658575dc-qvnpz\" (UID: \"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a\") " pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.842389 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.842338    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz"
Jul 31 12:21:31.971023 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:31.970999    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b658575dc-qvnpz"]
Jul 31 12:21:31.973430 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:21:31.973406    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74ca37b_1169_44ab_b8dc_7ea21b1c3b1a.slice/crio-9c77fc3e7dc6889a1535afe0ef951ef80adede3ef3fc06af1d37ab46ef56f134 WatchSource:0}: Error finding container 9c77fc3e7dc6889a1535afe0ef951ef80adede3ef3fc06af1d37ab46ef56f134: Status 404 returned error can't find the container with id 9c77fc3e7dc6889a1535afe0ef951ef80adede3ef3fc06af1d37ab46ef56f134
Jul 31 12:21:32.702334 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:32.702297    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz" event={"ID":"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a","Type":"ContainerStarted","Data":"9c77fc3e7dc6889a1535afe0ef951ef80adede3ef3fc06af1d37ab46ef56f134"}
Jul 31 12:21:34.713338 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.713304    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz" event={"ID":"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a","Type":"ContainerStarted","Data":"3c01d75103dd2a38271dc38d9a578703deb9228210ed790a5d439517a673dbdf"}
Jul 31 12:21:34.713699 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.713356    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz" event={"ID":"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a","Type":"ContainerStarted","Data":"55ebef3b48e5788239e5b61a0150293f08219cc7ec25e7d9c1264dd6867d410f"}
Jul 31 12:21:34.713699 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.713376    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz" event={"ID":"b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a","Type":"ContainerStarted","Data":"e08cf034530a724f5bca1a61e4ad1e4b2d32fc9793a587863a1cf907e52e5364"}
Jul 31 12:21:34.734673 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.734624    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b658575dc-qvnpz" podStartSLOduration=1.95102046 podStartE2EDuration="3.734609896s" totalImagesPullingTime="1.783589436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:21:34.733967207 +0000 UTC m=+381.965831985" watchObservedRunningTime="2026-07-31 12:21:34.734609896 +0000 UTC m=+381.966474646"
Jul 31 12:21:34.836298 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.836269    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:21:34.838573 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.838556    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.850683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.850658    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:21:34.886510 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886486    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886521    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886550    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886644    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886702    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886744    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.886849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.886824    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwk9\" (UniqueName: \"kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988140 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988066    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988140 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988112    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988140 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988136    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988188    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988236    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988752 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988712    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.988869 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.988812    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwk9\" (UniqueName: \"kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.989197 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.989147    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.989519 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.989467    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.994556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.989924    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.994556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.990027    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.995253 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.995233    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.995445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.995428    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:34.997196 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:34.997163    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwk9\" (UniqueName: \"kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9\") pod \"console-7bc5d4594-p68g4\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") " pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:35.147214 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:35.147154    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:35.272566 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:35.272542    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:21:35.274522 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:21:35.274493    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2f0995_77c8_4fd9_8f60_1814c708d9b1.slice/crio-bf68ddd8a2b8e3e0229f5d603a7cd5f0c04d661dbdf10ebb2c938399d7a9e77b WatchSource:0}: Error finding container bf68ddd8a2b8e3e0229f5d603a7cd5f0c04d661dbdf10ebb2c938399d7a9e77b: Status 404 returned error can't find the container with id bf68ddd8a2b8e3e0229f5d603a7cd5f0c04d661dbdf10ebb2c938399d7a9e77b
Jul 31 12:21:35.718867 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:35.718830    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc5d4594-p68g4" event={"ID":"5a2f0995-77c8-4fd9-8f60-1814c708d9b1","Type":"ContainerStarted","Data":"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"}
Jul 31 12:21:35.719273 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:35.718886    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc5d4594-p68g4" event={"ID":"5a2f0995-77c8-4fd9-8f60-1814c708d9b1","Type":"ContainerStarted","Data":"bf68ddd8a2b8e3e0229f5d603a7cd5f0c04d661dbdf10ebb2c938399d7a9e77b"}
Jul 31 12:21:35.734834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:35.734786    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bc5d4594-p68g4" podStartSLOduration=1.7347708960000001 podStartE2EDuration="1.734770896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:21:35.734127557 +0000 UTC m=+382.965992309" watchObservedRunningTime="2026-07-31 12:21:35.734770896 +0000 UTC m=+382.966635647"
Jul 31 12:21:45.147435 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.147403    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:45.147793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.147462    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:45.151861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.151840    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:45.764297 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.764267    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:21:45.809865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.809837    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:21:45.872696 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.872660    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:21:45.875829 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.875794    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.887828 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.887804    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:21:45.976396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976372    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976404    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdm6z\" (UniqueName: \"kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976423    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976444    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976639 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976534    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976639 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976599    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:45.976639 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:45.976630    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077291 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077244    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077291 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077276    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077421 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077302    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077518 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077498    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077569 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077555    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077653 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077597    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.077653 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.077630    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdm6z\" (UniqueName: \"kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.078093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.078052    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.078287 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.078264    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.078287 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.078277    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.078509 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.078485    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.079745 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.079727    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.079854 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.079836    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.084867 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.084839    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdm6z\" (UniqueName: \"kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z\") pod \"console-5f7dcb6dd7-h72hl\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") " pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.184374 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.184353    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:46.328971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.328944    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:21:46.331412 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:21:46.331382    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod371a2595_c1fe_4086_a6e2_341fb5030832.slice/crio-eee20a0643c3645c7352ab19020e02b17a2b4043aaa15908802e204f795d1fca WatchSource:0}: Error finding container eee20a0643c3645c7352ab19020e02b17a2b4043aaa15908802e204f795d1fca: Status 404 returned error can't find the container with id eee20a0643c3645c7352ab19020e02b17a2b4043aaa15908802e204f795d1fca
Jul 31 12:21:46.766071 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.766038    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcb6dd7-h72hl" event={"ID":"371a2595-c1fe-4086-a6e2-341fb5030832","Type":"ContainerStarted","Data":"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"}
Jul 31 12:21:46.766245 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.766085    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcb6dd7-h72hl" event={"ID":"371a2595-c1fe-4086-a6e2-341fb5030832","Type":"ContainerStarted","Data":"eee20a0643c3645c7352ab19020e02b17a2b4043aaa15908802e204f795d1fca"}
Jul 31 12:21:46.782716 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:46.782673    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f7dcb6dd7-h72hl" podStartSLOduration=1.782657208 podStartE2EDuration="1.782657208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:21:46.781847615 +0000 UTC m=+394.013712360" watchObservedRunningTime="2026-07-31 12:21:46.782657208 +0000 UTC m=+394.014521958"
Jul 31 12:21:56.184805 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:56.184768    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:56.185300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:56.184832    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:56.188988 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:56.188967    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:56.817919 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:56.817893    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:21:56.866093 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:21:56.866064    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:22:10.833497 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:10.833445    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c7fd98574-bdstw" podUID="d9910e09-8388-432d-9921-835006af57ae" containerName="console" containerID="cri-o://e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f" gracePeriod=15
Jul 31 12:22:11.070828 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.070804    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:22:11.179438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179343    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179386    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179410    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppc28\" (UniqueName: \"kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179430    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179453    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179479    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179639    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca\") pod \"d9910e09-8388-432d-9921-835006af57ae\" (UID: \"d9910e09-8388-432d-9921-835006af57ae\") "
Jul 31 12:22:11.179982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179846    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config" (OuterVolumeSpecName: "console-config") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:11.179982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179857    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:11.179982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.179897    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:11.180155 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.180137    2532 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.180225 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.180160    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.180225 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.180160    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:11.180225 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.180199    2532 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.181619 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.181597    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:22:11.181995 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.181977    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:22:11.182047 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.181990    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28" (OuterVolumeSpecName: "kube-api-access-ppc28") pod "d9910e09-8388-432d-9921-835006af57ae" (UID: "d9910e09-8388-432d-9921-835006af57ae"). InnerVolumeSpecName "kube-api-access-ppc28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:22:11.281223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.281156    2532 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.281223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.281213    2532 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9910e09-8388-432d-9921-835006af57ae-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.281223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.281224    2532 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9910e09-8388-432d-9921-835006af57ae-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.281223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.281233    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppc28\" (UniqueName: \"kubernetes.io/projected/d9910e09-8388-432d-9921-835006af57ae-kube-api-access-ppc28\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:11.885005 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.884970    2532 generic.go:356] "Generic (PLEG): container finished" podID="d9910e09-8388-432d-9921-835006af57ae" containerID="e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f" exitCode=0
Jul 31 12:22:11.885375 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.885016    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fd98574-bdstw"
Jul 31 12:22:11.885375 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.885041    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fd98574-bdstw" event={"ID":"d9910e09-8388-432d-9921-835006af57ae","Type":"ContainerDied","Data":"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"}
Jul 31 12:22:11.885375 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.885087    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fd98574-bdstw" event={"ID":"d9910e09-8388-432d-9921-835006af57ae","Type":"ContainerDied","Data":"cdfa50cc45a61136f07b5142753f2a77f2de0b2a0703e102b800e5cea77dbd7f"}
Jul 31 12:22:11.885375 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.885115    2532 scope.go:117] "RemoveContainer" containerID="e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"
Jul 31 12:22:11.893440 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.893420    2532 scope.go:117] "RemoveContainer" containerID="e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"
Jul 31 12:22:11.893670 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:22:11.893651    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f\": container with ID starting with e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f not found: ID does not exist" containerID="e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"
Jul 31 12:22:11.893715 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.893674    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f"} err="failed to get container status \"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f\": rpc error: code = NotFound desc = could not find container \"e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f\": container with ID starting with e52aed15558441b968e3e036feb3168df850e89f31129361982f10072aa68f8f not found: ID does not exist"
Jul 31 12:22:11.900910 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.900893    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:22:11.904306 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:11.904284    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c7fd98574-bdstw"]
Jul 31 12:22:13.346739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:13.346702    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9910e09-8388-432d-9921-835006af57ae" path="/var/lib/kubelet/pods/d9910e09-8388-432d-9921-835006af57ae/volumes"
Jul 31 12:22:21.884482 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:21.884445    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bc5d4594-p68g4" podUID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" containerName="console" containerID="cri-o://2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a" gracePeriod=15
Jul 31 12:22:22.115650 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.115628    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:22:22.161631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161567    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161602    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161629    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161820 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161654    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161820 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161803    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtwk9\" (UniqueName: \"kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161908 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161893    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.161952 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161940    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert\") pod \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\" (UID: \"5a2f0995-77c8-4fd9-8f60-1814c708d9b1\") "
Jul 31 12:22:22.162024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161993    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:22.162103 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.161997    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:22.162103 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.162054    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config" (OuterVolumeSpecName: "console-config") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:22.162245 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.162227    2532 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.162306 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.162247    2532 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.162306 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.162254    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.162394 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.162322    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:22:22.163860 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.163841    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:22:22.164111 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.164084    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:22:22.164227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.164087    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9" (OuterVolumeSpecName: "kube-api-access-rtwk9") pod "5a2f0995-77c8-4fd9-8f60-1814c708d9b1" (UID: "5a2f0995-77c8-4fd9-8f60-1814c708d9b1"). InnerVolumeSpecName "kube-api-access-rtwk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:22:22.262548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.262515    2532 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.262548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.262545    2532 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.262548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.262555    2532 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.262708 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.262564    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtwk9\" (UniqueName: \"kubernetes.io/projected/5a2f0995-77c8-4fd9-8f60-1814c708d9b1-kube-api-access-rtwk9\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:22:22.929600 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.929574    2532 generic.go:356] "Generic (PLEG): container finished" podID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" containerID="2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a" exitCode=0
Jul 31 12:22:22.929983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.929614    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc5d4594-p68g4"
Jul 31 12:22:22.929983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.929663    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc5d4594-p68g4" event={"ID":"5a2f0995-77c8-4fd9-8f60-1814c708d9b1","Type":"ContainerDied","Data":"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"}
Jul 31 12:22:22.929983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.929710    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc5d4594-p68g4" event={"ID":"5a2f0995-77c8-4fd9-8f60-1814c708d9b1","Type":"ContainerDied","Data":"bf68ddd8a2b8e3e0229f5d603a7cd5f0c04d661dbdf10ebb2c938399d7a9e77b"}
Jul 31 12:22:22.929983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.929730    2532 scope.go:117] "RemoveContainer" containerID="2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"
Jul 31 12:22:22.938061 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.938046    2532 scope.go:117] "RemoveContainer" containerID="2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"
Jul 31 12:22:22.938323 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:22:22.938304    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a\": container with ID starting with 2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a not found: ID does not exist" containerID="2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"
Jul 31 12:22:22.938383 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.938330    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a"} err="failed to get container status \"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a\": rpc error: code = NotFound desc = could not find container \"2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a\": container with ID starting with 2d59056ba9b5e01f3be7c411202a68530ce3ace749e0ff10482b2d34b1604c3a not found: ID does not exist"
Jul 31 12:22:22.948723 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.948703    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:22:22.952322 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:22.952303    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bc5d4594-p68g4"]
Jul 31 12:22:23.345295 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:22:23.345268    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" path="/var/lib/kubelet/pods/5a2f0995-77c8-4fd9-8f60-1814c708d9b1/volumes"
Jul 31 12:23:01.987697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.987616    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"]
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.987978    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" containerName="console"
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.987991    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" containerName="console"
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.988005    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9910e09-8388-432d-9921-835006af57ae" containerName="console"
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.988011    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9910e09-8388-432d-9921-835006af57ae" containerName="console"
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.988067    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9910e09-8388-432d-9921-835006af57ae" containerName="console"
Jul 31 12:23:01.988128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.988076    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2f0995-77c8-4fd9-8f60-1814c708d9b1" containerName="console"
Jul 31 12:23:01.990056 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.990039    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:01.992657 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.992632    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 31 12:23:01.992782 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.992692    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 31 12:23:01.992782 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.992728    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ndcnp\""
Jul 31 12:23:01.999305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:01.999285    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"]
Jul 31 12:23:02.045683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.045661    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.045795 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.045722    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.045795 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.045757    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn5jm\" (UniqueName: \"kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.146374 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.146346    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.146516 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.146406    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.146516 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.146443    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn5jm\" (UniqueName: \"kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.146760 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.146739    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.146798 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.146745    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.154282 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.154260    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn5jm\" (UniqueName: \"kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.298932 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.298856    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:02.422608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:02.422584    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"]
Jul 31 12:23:02.424420 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:02.424379    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ebd374_291b_4369_b611_db168f396e60.slice/crio-ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b WatchSource:0}: Error finding container ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b: Status 404 returned error can't find the container with id ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b
Jul 31 12:23:03.091935 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:03.091901    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm" event={"ID":"a9ebd374-291b-4369-b611-db168f396e60","Type":"ContainerStarted","Data":"ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b"}
Jul 31 12:23:11.130738 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:11.130699    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9ebd374-291b-4369-b611-db168f396e60" containerID="09bb8e67d73f5d8dbd2e22f84ce71af58cb602973ff9d342567f41e7d9fa6135" exitCode=0
Jul 31 12:23:11.131217 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:11.130785    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm" event={"ID":"a9ebd374-291b-4369-b611-db168f396e60","Type":"ContainerDied","Data":"09bb8e67d73f5d8dbd2e22f84ce71af58cb602973ff9d342567f41e7d9fa6135"}
Jul 31 12:23:13.140527 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:13.140441    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9ebd374-291b-4369-b611-db168f396e60" containerID="9c6dcc61916f8823631bcf18c0115283250be45d909107b1380d849dcf27ebaf" exitCode=0
Jul 31 12:23:13.140859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:13.140539    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm" event={"ID":"a9ebd374-291b-4369-b611-db168f396e60","Type":"ContainerDied","Data":"9c6dcc61916f8823631bcf18c0115283250be45d909107b1380d849dcf27ebaf"}
Jul 31 12:23:21.181484 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:21.181450    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9ebd374-291b-4369-b611-db168f396e60" containerID="f837b8a17f10db6964d512663a36da91070ca60d41855f1160c45df045d2b4b4" exitCode=0
Jul 31 12:23:21.181873 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:21.181535    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm" event={"ID":"a9ebd374-291b-4369-b611-db168f396e60","Type":"ContainerDied","Data":"f837b8a17f10db6964d512663a36da91070ca60d41855f1160c45df045d2b4b4"}
Jul 31 12:23:22.306210 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.306156    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:22.429379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.429352    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle\") pod \"a9ebd374-291b-4369-b611-db168f396e60\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") "
Jul 31 12:23:22.429549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.429415    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn5jm\" (UniqueName: \"kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm\") pod \"a9ebd374-291b-4369-b611-db168f396e60\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") "
Jul 31 12:23:22.429549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.429478    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util\") pod \"a9ebd374-291b-4369-b611-db168f396e60\" (UID: \"a9ebd374-291b-4369-b611-db168f396e60\") "
Jul 31 12:23:22.430048 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.430020    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle" (OuterVolumeSpecName: "bundle") pod "a9ebd374-291b-4369-b611-db168f396e60" (UID: "a9ebd374-291b-4369-b611-db168f396e60"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:22.431570 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.431543    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm" (OuterVolumeSpecName: "kube-api-access-bn5jm") pod "a9ebd374-291b-4369-b611-db168f396e60" (UID: "a9ebd374-291b-4369-b611-db168f396e60"). InnerVolumeSpecName "kube-api-access-bn5jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:23:22.436869 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.436845    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util" (OuterVolumeSpecName: "util") pod "a9ebd374-291b-4369-b611-db168f396e60" (UID: "a9ebd374-291b-4369-b611-db168f396e60"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:22.530249 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.530221    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:22.530249 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.530246    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn5jm\" (UniqueName: \"kubernetes.io/projected/a9ebd374-291b-4369-b611-db168f396e60-kube-api-access-bn5jm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:22.530397 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:22.530256    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9ebd374-291b-4369-b611-db168f396e60-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:23.190832 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:23.190799    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm"
Jul 31 12:23:23.190832 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:23.190824    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5jgm" event={"ID":"a9ebd374-291b-4369-b611-db168f396e60","Type":"ContainerDied","Data":"ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b"}
Jul 31 12:23:23.191067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:23.190866    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccce75da009b2702565e8f7b523077195e87de5913470be265034b88a6145f1b"
Jul 31 12:23:30.240965 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.240931    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"]
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241266    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="pull"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241278    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="pull"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241299    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="extract"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241305    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="extract"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241313    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="util"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241318    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="util"
Jul 31 12:23:30.241445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.241368    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9ebd374-291b-4369-b611-db168f396e60" containerName="extract"
Jul 31 12:23:30.244904 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.244888    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.247426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.247405    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-w7scw\""
Jul 31 12:23:30.247426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.247421    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:23:30.247574 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.247422    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 31 12:23:30.258199 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.258161    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"]
Jul 31 12:23:30.286653 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.286617    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d9de85f-576a-4e7c-982a-6794e7992c78-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.286779 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.286678    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb5zc\" (UniqueName: \"kubernetes.io/projected/9d9de85f-576a-4e7c-982a-6794e7992c78-kube-api-access-cb5zc\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.387674 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.387645    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d9de85f-576a-4e7c-982a-6794e7992c78-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.387823 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.387684    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb5zc\" (UniqueName: \"kubernetes.io/projected/9d9de85f-576a-4e7c-982a-6794e7992c78-kube-api-access-cb5zc\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.387999 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.387980    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d9de85f-576a-4e7c-982a-6794e7992c78-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.399214 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.399192    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb5zc\" (UniqueName: \"kubernetes.io/projected/9d9de85f-576a-4e7c-982a-6794e7992c78-kube-api-access-cb5zc\") pod \"cert-manager-operator-controller-manager-55c444bf86-w98p9\" (UID: \"9d9de85f-576a-4e7c-982a-6794e7992c78\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.553750 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.553670    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"
Jul 31 12:23:30.689407 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:30.689370    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9"]
Jul 31 12:23:30.692859 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:30.692831    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9de85f_576a_4e7c_982a_6794e7992c78.slice/crio-792aa80463c4af66428c9a0bca83ede941bb4ab27e94bd99929bab3dce259543 WatchSource:0}: Error finding container 792aa80463c4af66428c9a0bca83ede941bb4ab27e94bd99929bab3dce259543: Status 404 returned error can't find the container with id 792aa80463c4af66428c9a0bca83ede941bb4ab27e94bd99929bab3dce259543
Jul 31 12:23:31.225992 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:31.225957    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9" event={"ID":"9d9de85f-576a-4e7c-982a-6794e7992c78","Type":"ContainerStarted","Data":"792aa80463c4af66428c9a0bca83ede941bb4ab27e94bd99929bab3dce259543"}
Jul 31 12:23:33.237796 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:33.237759    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9" event={"ID":"9d9de85f-576a-4e7c-982a-6794e7992c78","Type":"ContainerStarted","Data":"af48b1d19f9092141e97a9b4218d4e79f79b5416d38e76fa9e350d35eb0933de"}
Jul 31 12:23:33.258767 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:33.258720    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w98p9" podStartSLOduration=1.679306341 podStartE2EDuration="3.258701955s" totalImagesPullingTime="1.579395614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:23:33.256796727 +0000 UTC m=+500.488661477" watchObservedRunningTime="2026-07-31 12:23:33.258701955 +0000 UTC m=+500.490566705"
Jul 31 12:23:34.656659 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.656623    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"]
Jul 31 12:23:34.658926 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.658910    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.661406 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.661384    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 31 12:23:34.661406 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.661403    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ndcnp\""
Jul 31 12:23:34.661545 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.661446    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 31 12:23:34.668797 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.668772    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"]
Jul 31 12:23:34.723237 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.723149    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpbbn\" (UniqueName: \"kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.723392 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.723246    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.723392 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.723342    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.824595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.824557    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.824595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.824598    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpbbn\" (UniqueName: \"kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.824823 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.824620    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.830042 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.825299    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.830042 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.825377    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.832797 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.832772    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpbbn\" (UniqueName: \"kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:34.968632 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:34.968604    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:35.094083 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:35.094057    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"]
Jul 31 12:23:35.096370 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:35.096342    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc94a929c_4638_4802_abe5_e8035a48b91a.slice/crio-ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920 WatchSource:0}: Error finding container ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920: Status 404 returned error can't find the container with id ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920
Jul 31 12:23:35.247835 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:35.247756    2532 generic.go:356] "Generic (PLEG): container finished" podID="c94a929c-4638-4802-abe5-e8035a48b91a" containerID="6b866cc9f8addb2b32314f0b4656251d6261d98504f33ee8664ab7965d8bbb16" exitCode=0
Jul 31 12:23:35.247975 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:35.247848    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb" event={"ID":"c94a929c-4638-4802-abe5-e8035a48b91a","Type":"ContainerDied","Data":"6b866cc9f8addb2b32314f0b4656251d6261d98504f33ee8664ab7965d8bbb16"}
Jul 31 12:23:35.247975 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:35.247907    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb" event={"ID":"c94a929c-4638-4802-abe5-e8035a48b91a","Type":"ContainerStarted","Data":"ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920"}
Jul 31 12:23:36.527073 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.527036    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"]
Jul 31 12:23:36.529644 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.529619    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.530102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.530074    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8z4b"]
Jul 31 12:23:36.531788 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.531768    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 31 12:23:36.532506 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.532484    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 31 12:23:36.532606 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.532504    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8c5lr\""
Jul 31 12:23:36.532879 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.532862    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.534711 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.534693    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m24mv\""
Jul 31 12:23:36.539376 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.539352    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"]
Jul 31 12:23:36.541992 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.541972    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8z4b"]
Jul 31 12:23:36.642739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.642700    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnbck\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-kube-api-access-cnbck\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.642899 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.642775    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl98v\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-kube-api-access-fl98v\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.642899 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.642836    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.642984 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.642915    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.744036 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.743983    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.744246 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.744062    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.744246 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.744101    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnbck\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-kube-api-access-cnbck\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.744246 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.744163    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl98v\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-kube-api-access-fl98v\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.752642 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.752607    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.752803 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.752674    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.752803 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.752771    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnbck\" (UniqueName: \"kubernetes.io/projected/beb08001-722d-4e8f-b204-09d861a53ab3-kube-api-access-cnbck\") pod \"cert-manager-cainjector-559cdd8c87-plz9b\" (UID: \"beb08001-722d-4e8f-b204-09d861a53ab3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.753099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.753055    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl98v\" (UniqueName: \"kubernetes.io/projected/db1cdd65-40c5-4f75-85db-067913cc05c2-kube-api-access-fl98v\") pod \"cert-manager-webhook-6486ffd997-l8z4b\" (UID: \"db1cdd65-40c5-4f75-85db-067913cc05c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:36.862132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.862059    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"
Jul 31 12:23:36.867490 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:36.867459    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:37.406004 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:37.405972    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8z4b"]
Jul 31 12:23:37.409594 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:37.409566    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb1cdd65_40c5_4f75_85db_067913cc05c2.slice/crio-2c36e2906bf96c2a2222059b5e8987580aaf21329121cf2573304b77511f8b08 WatchSource:0}: Error finding container 2c36e2906bf96c2a2222059b5e8987580aaf21329121cf2573304b77511f8b08: Status 404 returned error can't find the container with id 2c36e2906bf96c2a2222059b5e8987580aaf21329121cf2573304b77511f8b08
Jul 31 12:23:37.433324 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:37.433301    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-plz9b"]
Jul 31 12:23:37.451004 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:37.450978    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeb08001_722d_4e8f_b204_09d861a53ab3.slice/crio-933b8c559a0a81794031d80f5132f5c828b0fad0297d033e464ddba57b15421b WatchSource:0}: Error finding container 933b8c559a0a81794031d80f5132f5c828b0fad0297d033e464ddba57b15421b: Status 404 returned error can't find the container with id 933b8c559a0a81794031d80f5132f5c828b0fad0297d033e464ddba57b15421b
Jul 31 12:23:38.269807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:38.269740    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b" event={"ID":"beb08001-722d-4e8f-b204-09d861a53ab3","Type":"ContainerStarted","Data":"933b8c559a0a81794031d80f5132f5c828b0fad0297d033e464ddba57b15421b"}
Jul 31 12:23:38.271461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:38.271420    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b" event={"ID":"db1cdd65-40c5-4f75-85db-067913cc05c2","Type":"ContainerStarted","Data":"2c36e2906bf96c2a2222059b5e8987580aaf21329121cf2573304b77511f8b08"}
Jul 31 12:23:38.273826 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:38.273799    2532 generic.go:356] "Generic (PLEG): container finished" podID="c94a929c-4638-4802-abe5-e8035a48b91a" containerID="c896db4d6d9bd1063f3549d5dffc9a054459c0a2cec9c46bb2979e4ad821c1b3" exitCode=0
Jul 31 12:23:38.273929 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:38.273844    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb" event={"ID":"c94a929c-4638-4802-abe5-e8035a48b91a","Type":"ContainerDied","Data":"c896db4d6d9bd1063f3549d5dffc9a054459c0a2cec9c46bb2979e4ad821c1b3"}
Jul 31 12:23:39.280976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:39.280935    2532 generic.go:356] "Generic (PLEG): container finished" podID="c94a929c-4638-4802-abe5-e8035a48b91a" containerID="52dbc0bfba1ea6c914294356490dd3d7b2ada7778ee3887a26a298a865df2140" exitCode=0
Jul 31 12:23:39.281454 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:39.281032    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb" event={"ID":"c94a929c-4638-4802-abe5-e8035a48b91a","Type":"ContainerDied","Data":"52dbc0bfba1ea6c914294356490dd3d7b2ada7778ee3887a26a298a865df2140"}
Jul 31 12:23:40.287596 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.287558    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b" event={"ID":"beb08001-722d-4e8f-b204-09d861a53ab3","Type":"ContainerStarted","Data":"18ab73fb3b23afb432e9d1509c4e543d1f56449d3811dd534f1a6a9cf4a7706f"}
Jul 31 12:23:40.289181 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.289135    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b" event={"ID":"db1cdd65-40c5-4f75-85db-067913cc05c2","Type":"ContainerStarted","Data":"052c5aebd55985486e49a84330a093a9cb7e88eec97560812ac79d080eddc511"}
Jul 31 12:23:40.289294 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.289191    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:40.306896 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.306844    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-plz9b" podStartSLOduration=1.866487911 podStartE2EDuration="4.306825772s" totalImagesPullingTime="2.440337861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:23:40.305462551 +0000 UTC m=+507.537327302" watchObservedRunningTime="2026-07-31 12:23:40.306825772 +0000 UTC m=+507.538690522"
Jul 31 12:23:40.322221 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.322150    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b" podStartSLOduration=1.835043514 podStartE2EDuration="4.322132234s" totalImagesPullingTime="2.48708872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:23:40.321737724 +0000 UTC m=+507.553602475" watchObservedRunningTime="2026-07-31 12:23:40.322132234 +0000 UTC m=+507.553996985"
Jul 31 12:23:40.428323 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.428299    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:40.480125 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.480092    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpbbn\" (UniqueName: \"kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn\") pod \"c94a929c-4638-4802-abe5-e8035a48b91a\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") "
Jul 31 12:23:40.480265 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.480132    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util\") pod \"c94a929c-4638-4802-abe5-e8035a48b91a\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") "
Jul 31 12:23:40.480265 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.480151    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle\") pod \"c94a929c-4638-4802-abe5-e8035a48b91a\" (UID: \"c94a929c-4638-4802-abe5-e8035a48b91a\") "
Jul 31 12:23:40.480615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.480582    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle" (OuterVolumeSpecName: "bundle") pod "c94a929c-4638-4802-abe5-e8035a48b91a" (UID: "c94a929c-4638-4802-abe5-e8035a48b91a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:40.482326 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.482304    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn" (OuterVolumeSpecName: "kube-api-access-cpbbn") pod "c94a929c-4638-4802-abe5-e8035a48b91a" (UID: "c94a929c-4638-4802-abe5-e8035a48b91a"). InnerVolumeSpecName "kube-api-access-cpbbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:23:40.484315 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.484272    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util" (OuterVolumeSpecName: "util") pod "c94a929c-4638-4802-abe5-e8035a48b91a" (UID: "c94a929c-4638-4802-abe5-e8035a48b91a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:40.581629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.581542    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpbbn\" (UniqueName: \"kubernetes.io/projected/c94a929c-4638-4802-abe5-e8035a48b91a-kube-api-access-cpbbn\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:40.581629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.581571    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:40.581629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:40.581580    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c94a929c-4638-4802-abe5-e8035a48b91a-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:41.294946 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:41.294917    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb"
Jul 31 12:23:41.294946 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:41.294922    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd5snb" event={"ID":"c94a929c-4638-4802-abe5-e8035a48b91a","Type":"ContainerDied","Data":"ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920"}
Jul 31 12:23:41.295373 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:41.294974    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccec59c32546ace21d0af11bcc7d43c75f33623b3b5d8a7fc296c77dd0605920"
Jul 31 12:23:46.297305 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.297276    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-l8z4b"
Jul 31 12:23:46.555435 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555361    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"]
Jul 31 12:23:46.555759 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555745    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="extract"
Jul 31 12:23:46.555799 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555761    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="extract"
Jul 31 12:23:46.555799 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555784    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="pull"
Jul 31 12:23:46.555799 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555792    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="pull"
Jul 31 12:23:46.555922 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555805    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="util"
Jul 31 12:23:46.555922 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555811    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="util"
Jul 31 12:23:46.555922 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.555871    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="c94a929c-4638-4802-abe5-e8035a48b91a" containerName="extract"
Jul 31 12:23:46.557790 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.557772    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.560280 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.560256    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9btvm\""
Jul 31 12:23:46.560414 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.560293    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 31 12:23:46.560414 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.560301    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 31 12:23:46.566486 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.566462    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"]
Jul 31 12:23:46.633654 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.633622    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.633812 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.633663    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z4df\" (UniqueName: \"kubernetes.io/projected/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-kube-api-access-7z4df\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.734634 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.734588    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.734780 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.734641    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z4df\" (UniqueName: \"kubernetes.io/projected/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-kube-api-access-7z4df\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.734977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.734957    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.742939 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.742918    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z4df\" (UniqueName: \"kubernetes.io/projected/f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4-kube-api-access-7z4df\") pod \"openshift-lws-operator-fd8ccff4c-4l8gc\" (UID: \"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:46.868230 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:46.868116    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"
Jul 31 12:23:47.001628 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:47.001598    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc"]
Jul 31 12:23:47.003517 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:47.003486    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1364e44_b10e_4c4b_b6cd_f8e456e9c2a4.slice/crio-8a492806acdd72da46d1d246926e38e8b54c89c51d7000a2e6c7aff23954eb11 WatchSource:0}: Error finding container 8a492806acdd72da46d1d246926e38e8b54c89c51d7000a2e6c7aff23954eb11: Status 404 returned error can't find the container with id 8a492806acdd72da46d1d246926e38e8b54c89c51d7000a2e6c7aff23954eb11
Jul 31 12:23:47.329360 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:47.329324    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc" event={"ID":"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4","Type":"ContainerStarted","Data":"8a492806acdd72da46d1d246926e38e8b54c89c51d7000a2e6c7aff23954eb11"}
Jul 31 12:23:50.349740 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:50.349701    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc" event={"ID":"f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4","Type":"ContainerStarted","Data":"a93caf0f66d822d3fb042ef7024840aae32a574f8815a63a16fc2dbae0b2a2c8"}
Jul 31 12:23:50.365690 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:50.365641    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4l8gc" podStartSLOduration=2.001213585 podStartE2EDuration="4.365626256s" totalImagesPullingTime="2.364412671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:23:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:23:50.364458913 +0000 UTC m=+517.596323677" watchObservedRunningTime="2026-07-31 12:23:50.365626256 +0000 UTC m=+517.597491006"
Jul 31 12:23:53.014607 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.014567    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"]
Jul 31 12:23:53.029588 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.029558    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"]
Jul 31 12:23:53.029748 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.029696    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.032182 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.032152    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 31 12:23:53.033092 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.033069    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 31 12:23:53.033207 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.033111    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ndcnp\""
Jul 31 12:23:53.092706 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.092674    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvhsw\" (UniqueName: \"kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.092852 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.092728    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.092852 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.092813    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.193252 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.193217    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvhsw\" (UniqueName: \"kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.193429 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.193264    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.193553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.193523    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.193712 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.193691    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.193849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.193822    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.201276 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.201254    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvhsw\" (UniqueName: \"kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.338713 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.338620    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:53.479313 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:53.479289    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"]
Jul 31 12:23:53.481244 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:53.481216    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cd9030_dbdb_421e_bced_75cedddcc71c.slice/crio-70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd WatchSource:0}: Error finding container 70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd: Status 404 returned error can't find the container with id 70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd
Jul 31 12:23:54.368572 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:54.368531    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerID="f2ed870100079f407ebe3e61533faa8e97a5ed1dfac17e421cd8840756c544d3" exitCode=0
Jul 31 12:23:54.368973 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:54.368624    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs" event={"ID":"a9cd9030-dbdb-421e-bced-75cedddcc71c","Type":"ContainerDied","Data":"f2ed870100079f407ebe3e61533faa8e97a5ed1dfac17e421cd8840756c544d3"}
Jul 31 12:23:54.368973 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:54.368677    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs" event={"ID":"a9cd9030-dbdb-421e-bced-75cedddcc71c","Type":"ContainerStarted","Data":"70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd"}
Jul 31 12:23:55.069724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.069700    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sn2s7"]
Jul 31 12:23:55.071812 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.071798    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.074191 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.074158    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8zrzl\""
Jul 31 12:23:55.085074 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.085048    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sn2s7"]
Jul 31 12:23:55.210241 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.210208    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlg7n\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-kube-api-access-nlg7n\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.210408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.210253    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.310922 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.310838    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlg7n\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-kube-api-access-nlg7n\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.310922 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.310887    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.319612 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.319576    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.319721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.319636    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlg7n\" (UniqueName: \"kubernetes.io/projected/4b135546-3580-4055-9531-26b73dfdb0e6-kube-api-access-nlg7n\") pod \"cert-manager-55b4d954d5-sn2s7\" (UID: \"4b135546-3580-4055-9531-26b73dfdb0e6\") " pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.374685 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.374658    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerID="db582db3889a43e37c05b21d00d89d949f4448370a84634d422090ba7d22c298" exitCode=0
Jul 31 12:23:55.375057 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.374749    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs" event={"ID":"a9cd9030-dbdb-421e-bced-75cedddcc71c","Type":"ContainerDied","Data":"db582db3889a43e37c05b21d00d89d949f4448370a84634d422090ba7d22c298"}
Jul 31 12:23:55.440843 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.440820    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sn2s7"
Jul 31 12:23:55.573160 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:55.573085    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sn2s7"]
Jul 31 12:23:55.576633 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:23:55.576610    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b135546_3580_4055_9531_26b73dfdb0e6.slice/crio-728dec2098206aa7a53912e26866eff8c0597815eb449cf43f35f9ff27613ac5 WatchSource:0}: Error finding container 728dec2098206aa7a53912e26866eff8c0597815eb449cf43f35f9ff27613ac5: Status 404 returned error can't find the container with id 728dec2098206aa7a53912e26866eff8c0597815eb449cf43f35f9ff27613ac5
Jul 31 12:23:56.381719 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:56.381684    2532 generic.go:356] "Generic (PLEG): container finished" podID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerID="087ffb1aecffb37273b2d7e2dfd22c394c3dc2ee6af7a341b8e64020a3fbdaae" exitCode=0
Jul 31 12:23:56.382147 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:56.381764    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs" event={"ID":"a9cd9030-dbdb-421e-bced-75cedddcc71c","Type":"ContainerDied","Data":"087ffb1aecffb37273b2d7e2dfd22c394c3dc2ee6af7a341b8e64020a3fbdaae"}
Jul 31 12:23:56.383272 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:56.383254    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sn2s7" event={"ID":"4b135546-3580-4055-9531-26b73dfdb0e6","Type":"ContainerStarted","Data":"70059f38ee2066fd84fc2b8ab0313f7501019971a2373cf97c8dbd5ed3ef6b1d"}
Jul 31 12:23:56.383328 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:56.383284    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sn2s7" event={"ID":"4b135546-3580-4055-9531-26b73dfdb0e6","Type":"ContainerStarted","Data":"728dec2098206aa7a53912e26866eff8c0597815eb449cf43f35f9ff27613ac5"}
Jul 31 12:23:56.412629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:56.412587    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sn2s7" podStartSLOduration=1.4125716449999999 podStartE2EDuration="1.412571645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:23:56.411830585 +0000 UTC m=+523.643695336" watchObservedRunningTime="2026-07-31 12:23:56.412571645 +0000 UTC m=+523.644436396"
Jul 31 12:23:57.517613 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.517590    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:57.632148 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.632115    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle\") pod \"a9cd9030-dbdb-421e-bced-75cedddcc71c\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") "
Jul 31 12:23:57.632318 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.632188    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvhsw\" (UniqueName: \"kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw\") pod \"a9cd9030-dbdb-421e-bced-75cedddcc71c\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") "
Jul 31 12:23:57.632318 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.632214    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util\") pod \"a9cd9030-dbdb-421e-bced-75cedddcc71c\" (UID: \"a9cd9030-dbdb-421e-bced-75cedddcc71c\") "
Jul 31 12:23:57.632857 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.632835    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle" (OuterVolumeSpecName: "bundle") pod "a9cd9030-dbdb-421e-bced-75cedddcc71c" (UID: "a9cd9030-dbdb-421e-bced-75cedddcc71c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:57.634272 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.634249    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw" (OuterVolumeSpecName: "kube-api-access-dvhsw") pod "a9cd9030-dbdb-421e-bced-75cedddcc71c" (UID: "a9cd9030-dbdb-421e-bced-75cedddcc71c"). InnerVolumeSpecName "kube-api-access-dvhsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:23:57.637809 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.637761    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util" (OuterVolumeSpecName: "util") pod "a9cd9030-dbdb-421e-bced-75cedddcc71c" (UID: "a9cd9030-dbdb-421e-bced-75cedddcc71c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:23:57.733039 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.733000    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:57.733039 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.733033    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvhsw\" (UniqueName: \"kubernetes.io/projected/a9cd9030-dbdb-421e-bced-75cedddcc71c-kube-api-access-dvhsw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:57.733039 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:57.733045    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9cd9030-dbdb-421e-bced-75cedddcc71c-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:23:58.394500 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:58.394473    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs"
Jul 31 12:23:58.394500 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:58.394483    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7gxgs" event={"ID":"a9cd9030-dbdb-421e-bced-75cedddcc71c","Type":"ContainerDied","Data":"70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd"}
Jul 31 12:23:58.394742 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:23:58.394525    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70baab91a442cf1ef1b878e3ec1930afbaec515f5bfb66ce141e9375daab9cfd"
Jul 31 12:24:09.742380 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742292    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"]
Jul 31 12:24:09.742826 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742801    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="pull"
Jul 31 12:24:09.742826 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742819    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="pull"
Jul 31 12:24:09.742942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742832    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="extract"
Jul 31 12:24:09.742942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742839    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="extract"
Jul 31 12:24:09.742942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742866    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="util"
Jul 31 12:24:09.742942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742875    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="util"
Jul 31 12:24:09.743120 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.742954    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9cd9030-dbdb-421e-bced-75cedddcc71c" containerName="extract"
Jul 31 12:24:09.745057 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.745037    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.747546 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.747524    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 31 12:24:09.747640 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.747606    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 31 12:24:09.747949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.747932    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 31 12:24:09.748067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.748048    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4n6bj\""
Jul 31 12:24:09.748355 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.748335    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 31 12:24:09.766015 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.765993    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"]
Jul 31 12:24:09.839933 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.839898    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-tmp\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.840099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.839944    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rm57\" (UniqueName: \"kubernetes.io/projected/6ae3afb5-942b-497f-8dec-c438bde3f551-kube-api-access-7rm57\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.840099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.839964    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-manifests\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.840099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.840031    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.840099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.840066    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.940865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.940829    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rm57\" (UniqueName: \"kubernetes.io/projected/6ae3afb5-942b-497f-8dec-c438bde3f551-kube-api-access-7rm57\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.940865 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.940864    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-manifests\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.941074 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.940887    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.941074 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.940905    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.941074 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.940992    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-tmp\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.941293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.941273    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-manifests\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.943302 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.943278    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ae3afb5-942b-497f-8dec-c438bde3f551-tmp\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.943464 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.943443    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.943521 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.943467    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ae3afb5-942b-497f-8dec-c438bde3f551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:09.948538 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:09.948521    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rm57\" (UniqueName: \"kubernetes.io/projected/6ae3afb5-942b-497f-8dec-c438bde3f551-kube-api-access-7rm57\") pod \"opendatahub-operator-controller-manager-5996b4548f-xt67k\" (UID: \"6ae3afb5-942b-497f-8dec-c438bde3f551\") " pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:10.055059 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:10.054974    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:10.186619 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:10.186594    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"]
Jul 31 12:24:10.188582 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:24:10.188529    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae3afb5_942b_497f_8dec_c438bde3f551.slice/crio-b5c7c1bbbe87599cd9a1f70a9f6faaa84e9cd75f6e60e1f4e3b7a33e7bc3b30b WatchSource:0}: Error finding container b5c7c1bbbe87599cd9a1f70a9f6faaa84e9cd75f6e60e1f4e3b7a33e7bc3b30b: Status 404 returned error can't find the container with id b5c7c1bbbe87599cd9a1f70a9f6faaa84e9cd75f6e60e1f4e3b7a33e7bc3b30b
Jul 31 12:24:10.461504 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:10.461466    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k" event={"ID":"6ae3afb5-942b-497f-8dec-c438bde3f551","Type":"ContainerStarted","Data":"b5c7c1bbbe87599cd9a1f70a9f6faaa84e9cd75f6e60e1f4e3b7a33e7bc3b30b"}
Jul 31 12:24:13.485184 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:13.485122    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k" event={"ID":"6ae3afb5-942b-497f-8dec-c438bde3f551","Type":"ContainerStarted","Data":"333b62a85ed3309db6735cefb34482eee645733a24f95d0b3dc5768ff70bca88"}
Jul 31 12:24:14.492851 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:14.492819    2532 generic.go:356] "Generic (PLEG): container finished" podID="6ae3afb5-942b-497f-8dec-c438bde3f551" containerID="333b62a85ed3309db6735cefb34482eee645733a24f95d0b3dc5768ff70bca88" exitCode=0
Jul 31 12:24:14.493297 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:14.492912    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k" event={"ID":"6ae3afb5-942b-497f-8dec-c438bde3f551","Type":"ContainerDied","Data":"333b62a85ed3309db6735cefb34482eee645733a24f95d0b3dc5768ff70bca88"}
Jul 31 12:24:15.499432 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:15.499401    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k" event={"ID":"6ae3afb5-942b-497f-8dec-c438bde3f551","Type":"ContainerStarted","Data":"feede786d3aae5a0af85fc37228270801c137d618c472368eb308680fb6a21e7"}
Jul 31 12:24:15.499840 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:15.499513    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:15.522801 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:15.522751    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k" podStartSLOduration=2.227430871 podStartE2EDuration="6.522738012s" totalImagesPullingTime="3.32544949s" totalInitContainerRuntime="969.857651ms" isStatefulPod=false podCreationTimestamp="2026-07-31 12:24:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:24:15.52162889 +0000 UTC m=+542.753493650" watchObservedRunningTime="2026-07-31 12:24:15.522738012 +0000 UTC m=+542.754602763"
Jul 31 12:24:26.506842 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:26.506806    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5996b4548f-xt67k"
Jul 31 12:24:36.500260 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.500224    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:36.506220 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.506196    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.509812 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.509781    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-m8htf\""
Jul 31 12:24:36.510039 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.510023    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 12:24:36.510227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.510209    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 12:24:36.510537 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.510519    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 31 12:24:36.538575 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.538543    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:36.560364 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560337    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560364 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560369    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560386    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560487    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560509    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560534    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560710 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560578    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dp7d\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560710 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560610    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.560787 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.560700    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661134 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661093    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661134 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661136    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661163    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661211    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661275    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661298    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661636 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661493    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661636 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661573    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dp7d\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661636 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661583    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661636 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661617    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661878 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661720    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.661965 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.661938    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.662108 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.662094    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.662255 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.662233    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.663540 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.663518    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.663807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.663789    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.671258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.671233    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.671458 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.671434    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dp7d\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck4hmv\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.816505 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.816417    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:36.954876 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:36.954843    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:36.958338 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:24:36.958303    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69676436_9ab9_4d9e_87de_9953dfc5bb66.slice/crio-7d510917c8b39dcfe5376981170220a56046c4b48fe7d5f8522986f9e0eab52d WatchSource:0}: Error finding container 7d510917c8b39dcfe5376981170220a56046c4b48fe7d5f8522986f9e0eab52d: Status 404 returned error can't find the container with id 7d510917c8b39dcfe5376981170220a56046c4b48fe7d5f8522986f9e0eab52d
Jul 31 12:24:37.603459 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:37.603418    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" event={"ID":"69676436-9ab9-4d9e-87de-9953dfc5bb66","Type":"ContainerStarted","Data":"7d510917c8b39dcfe5376981170220a56046c4b48fe7d5f8522986f9e0eab52d"}
Jul 31 12:24:39.605812 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.605749    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 31 12:24:39.606086 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.605869    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 31 12:24:39.606086 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.605899    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 31 12:24:39.697469 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.697424    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"]
Jul 31 12:24:39.700668 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.700643    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.706221 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.706193    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w8xbd\""
Jul 31 12:24:39.706564 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.706539    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 31 12:24:39.707153 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.707134    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 31 12:24:39.707635 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.707616    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 31 12:24:39.726419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.726391    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"]
Jul 31 12:24:39.793321 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.793288    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c4d0824d-6111-4585-980c-fd9f20ff597f-manager-config\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.793321 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.793320    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-metrics-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.793607 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.793346    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.793607 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.793363    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t9vr\" (UniqueName: \"kubernetes.io/projected/c4d0824d-6111-4585-980c-fd9f20ff597f-kube-api-access-9t9vr\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.894917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.894835    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c4d0824d-6111-4585-980c-fd9f20ff597f-manager-config\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.894917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.894869    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-metrics-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.894917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.894888    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.894917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.894908    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t9vr\" (UniqueName: \"kubernetes.io/projected/c4d0824d-6111-4585-980c-fd9f20ff597f-kube-api-access-9t9vr\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.895659 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.895635    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c4d0824d-6111-4585-980c-fd9f20ff597f-manager-config\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.897547 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.897523    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-metrics-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.897620 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.897572    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4d0824d-6111-4585-980c-fd9f20ff597f-cert\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:39.913631 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:39.913611    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t9vr\" (UniqueName: \"kubernetes.io/projected/c4d0824d-6111-4585-980c-fd9f20ff597f-kube-api-access-9t9vr\") pod \"lws-controller-manager-7f679cb99-gr9lx\" (UID: \"c4d0824d-6111-4585-980c-fd9f20ff597f\") " pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:40.012014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.011981    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:40.163808 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.163758    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"]
Jul 31 12:24:40.166115 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:24:40.166084    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d0824d_6111_4585_980c_fd9f20ff597f.slice/crio-0d5d48670d307f7c4254076b1d9639bc413a4eb19eaf33b1a5cb5f87806a440d WatchSource:0}: Error finding container 0d5d48670d307f7c4254076b1d9639bc413a4eb19eaf33b1a5cb5f87806a440d: Status 404 returned error can't find the container with id 0d5d48670d307f7c4254076b1d9639bc413a4eb19eaf33b1a5cb5f87806a440d
Jul 31 12:24:40.624368 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.624331    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" event={"ID":"69676436-9ab9-4d9e-87de-9953dfc5bb66","Type":"ContainerStarted","Data":"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"}
Jul 31 12:24:40.626133 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.626104    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx" event={"ID":"c4d0824d-6111-4585-980c-fd9f20ff597f","Type":"ContainerStarted","Data":"0d5d48670d307f7c4254076b1d9639bc413a4eb19eaf33b1a5cb5f87806a440d"}
Jul 31 12:24:40.653752 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.653702    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podStartSLOduration=2.008978991 podStartE2EDuration="4.653685446s" totalImagesPullingTime="2.644706455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:24:40.651984291 +0000 UTC m=+567.883849043" watchObservedRunningTime="2026-07-31 12:24:40.653685446 +0000 UTC m=+567.885550197"
Jul 31 12:24:40.717027 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.716994    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"]
Jul 31 12:24:40.720017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.720001    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.722357 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.722332    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 31 12:24:40.722460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.722386    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d7ld9\""
Jul 31 12:24:40.734548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.734518    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"]
Jul 31 12:24:40.803766 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.803729    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2c0d9e-cd8e-4612-932b-a3a72be75039-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.803919 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.803782    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbvtj\" (UniqueName: \"kubernetes.io/projected/cf2c0d9e-cd8e-4612-932b-a3a72be75039-kube-api-access-dbvtj\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.817154 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.817121    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:40.818591 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.818562    2532 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 31 12:24:40.818712 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.818613    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 31 12:24:40.904676 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.904591    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2c0d9e-cd8e-4612-932b-a3a72be75039-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.904676 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.904656    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbvtj\" (UniqueName: \"kubernetes.io/projected/cf2c0d9e-cd8e-4612-932b-a3a72be75039-kube-api-access-dbvtj\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.907430 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.907404    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2c0d9e-cd8e-4612-932b-a3a72be75039-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:40.923034 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:40.923006    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbvtj\" (UniqueName: \"kubernetes.io/projected/cf2c0d9e-cd8e-4612-932b-a3a72be75039-kube-api-access-dbvtj\") pod \"llmisvc-controller-manager-777c6bd7d5-mbvsh\" (UID: \"cf2c0d9e-cd8e-4612-932b-a3a72be75039\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:41.030140 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:41.030102    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:41.196697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:41.196663    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"]
Jul 31 12:24:41.198034 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:24:41.198008    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcf2c0d9e_cd8e_4612_932b_a3a72be75039.slice/crio-7d17efc13202cb79f4e1c45a2f78d10a0b08a678bcb2bc7f4673f61f5f1c4866 WatchSource:0}: Error finding container 7d17efc13202cb79f4e1c45a2f78d10a0b08a678bcb2bc7f4673f61f5f1c4866: Status 404 returned error can't find the container with id 7d17efc13202cb79f4e1c45a2f78d10a0b08a678bcb2bc7f4673f61f5f1c4866
Jul 31 12:24:41.632091 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:41.632058    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh" event={"ID":"cf2c0d9e-cd8e-4612-932b-a3a72be75039","Type":"ContainerStarted","Data":"7d17efc13202cb79f4e1c45a2f78d10a0b08a678bcb2bc7f4673f61f5f1c4866"}
Jul 31 12:24:41.817005 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:41.816945    2532 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 31 12:24:41.817242 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:41.817008    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 31 12:24:42.640373 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:42.640329    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx" event={"ID":"c4d0824d-6111-4585-980c-fd9f20ff597f","Type":"ContainerStarted","Data":"a82a86c46d2cbe6651e7481351ae032bb3af5eddb9072547697ceb9510865224"}
Jul 31 12:24:42.640911 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:42.640438    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:24:42.660079 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:42.660020    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx" podStartSLOduration=2.165574234 podStartE2EDuration="3.660002735s" totalImagesPullingTime="1.494428501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:24:42.658109708 +0000 UTC m=+569.889974461" watchObservedRunningTime="2026-07-31 12:24:42.660002735 +0000 UTC m=+569.891867486"
Jul 31 12:24:42.817292 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:42.817260    2532 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 31 12:24:42.817479 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:42.817312    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 31 12:24:43.816835 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:43.816798    2532 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 31 12:24:43.817204 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:43.816851    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 31 12:24:44.284945 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:44.284912    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:44.285137 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:44.285114    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy" containerID="cri-o://52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6" gracePeriod=30
Jul 31 12:24:44.654271 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:44.654193    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh" event={"ID":"cf2c0d9e-cd8e-4612-932b-a3a72be75039","Type":"ContainerStarted","Data":"f67b6192884f41575c354a97ad3557fc400e2d7d7145ff2db8aba5f5bff006cc"}
Jul 31 12:24:44.654437 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:44.654321    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:24:44.673369 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:44.673325    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh" podStartSLOduration=2.051983855 podStartE2EDuration="4.673311537s" totalImagesPullingTime="2.621327682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:24:44.671267068 +0000 UTC m=+571.903131819" watchObservedRunningTime="2026-07-31 12:24:44.673311537 +0000 UTC m=+571.905176287"
Jul 31 12:24:49.533772 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.533746    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:49.583898 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583828    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.583898 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583865    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.583898 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583887    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583911    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583945    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.583969    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584006    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584132 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584100    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584115    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:24:49.584408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584161    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:24:49.584408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584202    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dp7d\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d\") pod \"69676436-9ab9-4d9e-87de-9953dfc5bb66\" (UID: \"69676436-9ab9-4d9e-87de-9953dfc5bb66\") "
Jul 31 12:24:49.584408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584301    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:24:49.584408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584384    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data" (OuterVolumeSpecName: "istio-data") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584451    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584561    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-credential-socket\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584574    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584583    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69676436-9ab9-4d9e-87de-9953dfc5bb66-istiod-ca-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584590    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-workload-socket\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.584665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.584598    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-data\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.586311 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.586287    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token" (OuterVolumeSpecName: "istio-token") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:24:49.586452 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.586427    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d" (OuterVolumeSpecName: "kube-api-access-7dp7d") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "kube-api-access-7dp7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:24:49.586532 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.586513    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 31 12:24:49.586581 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.586567    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69676436-9ab9-4d9e-87de-9953dfc5bb66" (UID: "69676436-9ab9-4d9e-87de-9953dfc5bb66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:24:49.679605 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.679572    2532 generic.go:356] "Generic (PLEG): container finished" podID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerID="52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6" exitCode=0
Jul 31 12:24:49.679761 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.679617    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"
Jul 31 12:24:49.679761 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.679642    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" event={"ID":"69676436-9ab9-4d9e-87de-9953dfc5bb66","Type":"ContainerDied","Data":"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"}
Jul 31 12:24:49.679761 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.679685    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv" event={"ID":"69676436-9ab9-4d9e-87de-9953dfc5bb66","Type":"ContainerDied","Data":"7d510917c8b39dcfe5376981170220a56046c4b48fe7d5f8522986f9e0eab52d"}
Jul 31 12:24:49.679761 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.679709    2532 scope.go:117] "RemoveContainer" containerID="52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"
Jul 31 12:24:49.685312 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.685290    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dp7d\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-kube-api-access-7dp7d\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.685396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.685317    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-envoy\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.685396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.685331    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-podinfo\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.685396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.685342    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69676436-9ab9-4d9e-87de-9953dfc5bb66-istio-token\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:24:49.689647 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.689630    2532 scope.go:117] "RemoveContainer" containerID="52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"
Jul 31 12:24:49.689910 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:24:49.689886    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6\": container with ID starting with 52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6 not found: ID does not exist" containerID="52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"
Jul 31 12:24:49.689967 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.689922    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6"} err="failed to get container status \"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6\": rpc error: code = NotFound desc = could not find container \"52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6\": container with ID starting with 52d283f41dd0e4e4db0109742e5dc16279061a3ec06386a0a72a7e070d110ba6 not found: ID does not exist"
Jul 31 12:24:49.703157 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.703134    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:49.707859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:49.707837    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck4hmv"]
Jul 31 12:24:51.345851 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:51.345819    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" path="/var/lib/kubelet/pods/69676436-9ab9-4d9e-87de-9953dfc5bb66/volumes"
Jul 31 12:24:53.650138 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:24:53.650107    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f679cb99-gr9lx"
Jul 31 12:25:14.929551 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.929517    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-597b994885-fs958"]
Jul 31 12:25:14.961760 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.929878    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy"
Jul 31 12:25:14.961760 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.929889    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy"
Jul 31 12:25:14.961760 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.929958    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="69676436-9ab9-4d9e-87de-9953dfc5bb66" containerName="istio-proxy"
Jul 31 12:25:14.969704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.969682    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597b994885-fs958"]
Jul 31 12:25:14.969810 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:14.969776    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096081 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096044    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cp7n\" (UniqueName: \"kubernetes.io/projected/75a04fb6-f383-4c40-8a5c-f59e2b455239-kube-api-access-2cp7n\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096087    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096113    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-oauth-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096211    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-trusted-ca-bundle\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096251    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096475 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096323    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-oauth-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.096475 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.096352    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-service-ca\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197815 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197734    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197815 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197792    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-oauth-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197990 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197827    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-trusted-ca-bundle\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197990 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197854    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197990 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197901    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-oauth-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.197990 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.197929    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-service-ca\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.198193 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.198018    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cp7n\" (UniqueName: \"kubernetes.io/projected/75a04fb6-f383-4c40-8a5c-f59e2b455239-kube-api-access-2cp7n\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.198677 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.198652    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-oauth-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.198786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.198746    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-trusted-ca-bundle\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.198786 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.198747    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.198857 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.198806    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75a04fb6-f383-4c40-8a5c-f59e2b455239-service-ca\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.200700 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.200675    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-serving-cert\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.200789 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.200776    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75a04fb6-f383-4c40-8a5c-f59e2b455239-console-oauth-config\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.206060 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.206039    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cp7n\" (UniqueName: \"kubernetes.io/projected/75a04fb6-f383-4c40-8a5c-f59e2b455239-kube-api-access-2cp7n\") pod \"console-597b994885-fs958\" (UID: \"75a04fb6-f383-4c40-8a5c-f59e2b455239\") " pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.278923 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.278900    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:15.417851 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.417825    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597b994885-fs958"]
Jul 31 12:25:15.420732 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:25:15.420705    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a04fb6_f383_4c40_8a5c_f59e2b455239.slice/crio-86a497229f389d7318430e1c26022333bcceb201130c5600e6cb2bdc3ca4bdad WatchSource:0}: Error finding container 86a497229f389d7318430e1c26022333bcceb201130c5600e6cb2bdc3ca4bdad: Status 404 returned error can't find the container with id 86a497229f389d7318430e1c26022333bcceb201130c5600e6cb2bdc3ca4bdad
Jul 31 12:25:15.665764 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.665731    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mbvsh"
Jul 31 12:25:15.805906 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.805871    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597b994885-fs958" event={"ID":"75a04fb6-f383-4c40-8a5c-f59e2b455239","Type":"ContainerStarted","Data":"c91fd243625ae3f3515db9fcd7c752ff0ca7f49b87bcb4e33149c6618407f7d4"}
Jul 31 12:25:15.806073 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.805916    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597b994885-fs958" event={"ID":"75a04fb6-f383-4c40-8a5c-f59e2b455239","Type":"ContainerStarted","Data":"86a497229f389d7318430e1c26022333bcceb201130c5600e6cb2bdc3ca4bdad"}
Jul 31 12:25:15.823269 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:15.823215    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-597b994885-fs958" podStartSLOduration=1.8231934459999999 podStartE2EDuration="1.823193446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:25:15.821357292 +0000 UTC m=+603.053222054" watchObservedRunningTime="2026-07-31 12:25:15.823193446 +0000 UTC m=+603.055058200"
Jul 31 12:25:25.279635 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:25.279591    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:25.279995 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:25.279655    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:25.283987 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:25.283966    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:25.857379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:25.857345    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-597b994885-fs958"
Jul 31 12:25:25.903611 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:25.903583    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:25:33.330023 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.329983    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"]
Jul 31 12:25:33.333306 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.333285    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.335603 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.335583    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 31 12:25:33.335715 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.335579    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 31 12:25:33.336251 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.336233    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ndcnp\""
Jul 31 12:25:33.347316 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.347296    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"]
Jul 31 12:25:33.427490 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.427456    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"]
Jul 31 12:25:33.430358 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.430341    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.435967 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.435943    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.436073 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.436004    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.436545 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.436212    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pgqv\" (UniqueName: \"kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.438720 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.438698    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"]
Jul 31 12:25:33.530300 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.530264    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"]
Jul 31 12:25:33.533144 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.533129    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.536907 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.536884    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.537018 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.536920    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.537018 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.536948    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.537018 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.536972    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.537159 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.537037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pgqv\" (UniqueName: \"kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.537159 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.537062    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvb9c\" (UniqueName: \"kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.537362 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.537340    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.537405 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.537358    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.542677 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.542657    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"]
Jul 31 12:25:33.545939 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.545918    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pgqv\" (UniqueName: \"kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.630281 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.630209    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"]
Jul 31 12:25:33.632861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.632844    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.638270 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638234    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.638386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638275    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd4ms\" (UniqueName: \"kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.638386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638324    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.638386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638361    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvb9c\" (UniqueName: \"kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.638562 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638410    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.638562 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638499    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.638890 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.638864    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.639113 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.639087    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.641677 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.641656    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"]
Jul 31 12:25:33.643600 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.643582    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:33.646408 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.646384    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvb9c\" (UniqueName: \"kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.739809 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.739774    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd4ms\" (UniqueName: \"kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.739951 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.739831    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.739951 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.739864    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:33.739951 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.739882    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.740227 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.740205    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.740349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.740323    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.740349 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.740333    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.740445 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.740430    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cmqt\" (UniqueName: \"kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.740533 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.740511    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.748007 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.747988    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd4ms\" (UniqueName: \"kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.773340 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.773315    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"]
Jul 31 12:25:33.775337 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:25:33.775316    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa4e150_affe_4788_b6d2_631e1da6ba08.slice/crio-1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e WatchSource:0}: Error finding container 1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e: Status 404 returned error can't find the container with id 1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e
Jul 31 12:25:33.841499 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.841474    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cmqt\" (UniqueName: \"kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.841629 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.841613    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.841693 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.841643    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.841957 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.841936    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:33.842188 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.842093    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.842188 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.842145    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.850921 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.850899    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cmqt\" (UniqueName: \"kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.876823 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.876794    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"]
Jul 31 12:25:33.878484 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:25:33.878460    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0d1e76_683c_4dde_b2fd_179b722b6ae5.slice/crio-57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5 WatchSource:0}: Error finding container 57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5: Status 404 returned error can't find the container with id 57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5
Jul 31 12:25:33.894781 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.894748    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerStarted","Data":"2574c2ebdce33390b2c1b67f0de5a52201b5a5ebd6f19a5220b0dda46d76e4d3"}
Jul 31 12:25:33.894899 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.894812    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerStarted","Data":"1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e"}
Jul 31 12:25:33.895834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.895815    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 12:25:33.896215 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.896165    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk" event={"ID":"5b0d1e76-683c-4dde-b2fd-179b722b6ae5","Type":"ContainerStarted","Data":"57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5"}
Jul 31 12:25:33.942911 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.942889    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:33.986976 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:25:33.986939    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e8a9a9_bb68_4ecd_a2f7_e40062b8f263.slice/crio-198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d WatchSource:0}: Error finding container 198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d: Status 404 returned error can't find the container with id 198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d
Jul 31 12:25:33.987360 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:33.987330    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"]
Jul 31 12:25:34.081302 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.081281    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"]
Jul 31 12:25:34.088874 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:25:34.088847    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914af701_5f79_4e15_b8d2_7f563c6a28d0.slice/crio-bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0 WatchSource:0}: Error finding container bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0: Status 404 returned error can't find the container with id bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0
Jul 31 12:25:34.903343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.903314    2532 generic.go:356] "Generic (PLEG): container finished" podID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerID="2574c2ebdce33390b2c1b67f0de5a52201b5a5ebd6f19a5220b0dda46d76e4d3" exitCode=0
Jul 31 12:25:34.903343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.903343    2532 generic.go:356] "Generic (PLEG): container finished" podID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerID="95ad7fdb1f3ff2ff8e653b53e3e661c98ca3fd0981145bab2d0385166500e734" exitCode=0
Jul 31 12:25:34.903700 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.903400    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerDied","Data":"2574c2ebdce33390b2c1b67f0de5a52201b5a5ebd6f19a5220b0dda46d76e4d3"}
Jul 31 12:25:34.903700 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.903482    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerDied","Data":"95ad7fdb1f3ff2ff8e653b53e3e661c98ca3fd0981145bab2d0385166500e734"}
Jul 31 12:25:34.905007 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.904985    2532 generic.go:356] "Generic (PLEG): container finished" podID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerID="256fe953974af369263c77aac52b98ae9361532eb725299517d27e35e6e1878d" exitCode=0
Jul 31 12:25:34.905098 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.905050    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs" event={"ID":"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263","Type":"ContainerDied","Data":"256fe953974af369263c77aac52b98ae9361532eb725299517d27e35e6e1878d"}
Jul 31 12:25:34.905098 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.905093    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs" event={"ID":"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263","Type":"ContainerStarted","Data":"198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d"}
Jul 31 12:25:34.907083 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.907061    2532 generic.go:356] "Generic (PLEG): container finished" podID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerID="e005bf1f478abe5b7097f93786ba17ab7cc89f7bd72676ccc7eb1f9a13e1d69a" exitCode=0
Jul 31 12:25:34.907194 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.907089    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l" event={"ID":"914af701-5f79-4e15-b8d2-7f563c6a28d0","Type":"ContainerDied","Data":"e005bf1f478abe5b7097f93786ba17ab7cc89f7bd72676ccc7eb1f9a13e1d69a"}
Jul 31 12:25:34.907194 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.907122    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l" event={"ID":"914af701-5f79-4e15-b8d2-7f563c6a28d0","Type":"ContainerStarted","Data":"bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0"}
Jul 31 12:25:34.908771 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.908754    2532 generic.go:356] "Generic (PLEG): container finished" podID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerID="4b2a8a6156c7b11ba938de0eafea07b68ad97ceca6f44e6afd32a36ecd720938" exitCode=0
Jul 31 12:25:34.908855 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:34.908790    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk" event={"ID":"5b0d1e76-683c-4dde-b2fd-179b722b6ae5","Type":"ContainerDied","Data":"4b2a8a6156c7b11ba938de0eafea07b68ad97ceca6f44e6afd32a36ecd720938"}
Jul 31 12:25:35.915501 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.915474    2532 generic.go:356] "Generic (PLEG): container finished" podID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerID="de4626ab2f85e8aeded51a3d00c7ea8a5ad5fa897fbf99eb756fb2baf0f9f651" exitCode=0
Jul 31 12:25:35.915846 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.915525    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l" event={"ID":"914af701-5f79-4e15-b8d2-7f563c6a28d0","Type":"ContainerDied","Data":"de4626ab2f85e8aeded51a3d00c7ea8a5ad5fa897fbf99eb756fb2baf0f9f651"}
Jul 31 12:25:35.917399 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.917377    2532 generic.go:356] "Generic (PLEG): container finished" podID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerID="b29466d0c53a04905c1eaab07a8f767f7eca6752baa5e1eb2a2087dca1438a0f" exitCode=0
Jul 31 12:25:35.917493 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.917411    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk" event={"ID":"5b0d1e76-683c-4dde-b2fd-179b722b6ae5","Type":"ContainerDied","Data":"b29466d0c53a04905c1eaab07a8f767f7eca6752baa5e1eb2a2087dca1438a0f"}
Jul 31 12:25:35.919808 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.919788    2532 generic.go:356] "Generic (PLEG): container finished" podID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerID="4405ead77430045f2a51057ed25abe78e3b4c31d832bcc2d2bf25f489b5b8885" exitCode=0
Jul 31 12:25:35.919886 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:35.919868    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerDied","Data":"4405ead77430045f2a51057ed25abe78e3b4c31d832bcc2d2bf25f489b5b8885"}
Jul 31 12:25:36.926697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.926611    2532 generic.go:356] "Generic (PLEG): container finished" podID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerID="5d965abe05b0fe6c5902976f5208f21d7fd23e3cd9cd208e570f3196cbfa54da" exitCode=0
Jul 31 12:25:36.926697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.926687    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs" event={"ID":"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263","Type":"ContainerDied","Data":"5d965abe05b0fe6c5902976f5208f21d7fd23e3cd9cd208e570f3196cbfa54da"}
Jul 31 12:25:36.928986 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.928962    2532 generic.go:356] "Generic (PLEG): container finished" podID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerID="5f2f6f8b5676a5e98d1373497456f4759411eee2110ae29f0f394652a0c3eddd" exitCode=0
Jul 31 12:25:36.929068 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.929048    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l" event={"ID":"914af701-5f79-4e15-b8d2-7f563c6a28d0","Type":"ContainerDied","Data":"5f2f6f8b5676a5e98d1373497456f4759411eee2110ae29f0f394652a0c3eddd"}
Jul 31 12:25:36.931256 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.931236    2532 generic.go:356] "Generic (PLEG): container finished" podID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerID="44945613b3f50ac6b8deb0b20937b7ee706fdfe451a5b13adcd917fd74259685" exitCode=0
Jul 31 12:25:36.931317 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:36.931295    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk" event={"ID":"5b0d1e76-683c-4dde-b2fd-179b722b6ae5","Type":"ContainerDied","Data":"44945613b3f50ac6b8deb0b20937b7ee706fdfe451a5b13adcd917fd74259685"}
Jul 31 12:25:37.061303 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.061280    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:37.170870 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.170845    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle\") pod \"8fa4e150-affe-4788-b6d2-631e1da6ba08\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") "
Jul 31 12:25:37.170984 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.170919    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pgqv\" (UniqueName: \"kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv\") pod \"8fa4e150-affe-4788-b6d2-631e1da6ba08\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") "
Jul 31 12:25:37.171027 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.170984    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util\") pod \"8fa4e150-affe-4788-b6d2-631e1da6ba08\" (UID: \"8fa4e150-affe-4788-b6d2-631e1da6ba08\") "
Jul 31 12:25:37.171387 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.171361    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle" (OuterVolumeSpecName: "bundle") pod "8fa4e150-affe-4788-b6d2-631e1da6ba08" (UID: "8fa4e150-affe-4788-b6d2-631e1da6ba08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:37.172849 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.172830    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv" (OuterVolumeSpecName: "kube-api-access-2pgqv") pod "8fa4e150-affe-4788-b6d2-631e1da6ba08" (UID: "8fa4e150-affe-4788-b6d2-631e1da6ba08"). InnerVolumeSpecName "kube-api-access-2pgqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:25:37.176346 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.176321    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util" (OuterVolumeSpecName: "util") pod "8fa4e150-affe-4788-b6d2-631e1da6ba08" (UID: "8fa4e150-affe-4788-b6d2-631e1da6ba08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:37.271496 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.271473    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:37.271595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.271509    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fa4e150-affe-4788-b6d2-631e1da6ba08-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:37.271595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.271519    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pgqv\" (UniqueName: \"kubernetes.io/projected/8fa4e150-affe-4788-b6d2-631e1da6ba08-kube-api-access-2pgqv\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:37.938875 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.938786    2532 generic.go:356] "Generic (PLEG): container finished" podID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerID="7dc33bcc2ce464dd4eab14785a0726e15531864e0361129a1b00628c4e8ff2ac" exitCode=0
Jul 31 12:25:37.939361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.938869    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs" event={"ID":"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263","Type":"ContainerDied","Data":"7dc33bcc2ce464dd4eab14785a0726e15531864e0361129a1b00628c4e8ff2ac"}
Jul 31 12:25:37.940700 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.940672    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj" event={"ID":"8fa4e150-affe-4788-b6d2-631e1da6ba08","Type":"ContainerDied","Data":"1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e"}
Jul 31 12:25:37.940700 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.940693    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07g7jj"
Jul 31 12:25:37.940856 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:37.940714    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a1ead1d38c30a66f72a626cd46928c1ba115be2341bfcbbedab0e84de39899e"
Jul 31 12:25:38.068871 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.068848    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:38.100694 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.100661    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:38.179729 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179688    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cmqt\" (UniqueName: \"kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt\") pod \"914af701-5f79-4e15-b8d2-7f563c6a28d0\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") "
Jul 31 12:25:38.179729 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179736    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvb9c\" (UniqueName: \"kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c\") pod \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") "
Jul 31 12:25:38.180002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179769    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle\") pod \"914af701-5f79-4e15-b8d2-7f563c6a28d0\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") "
Jul 31 12:25:38.180002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179789    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util\") pod \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") "
Jul 31 12:25:38.180002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179809    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle\") pod \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\" (UID: \"5b0d1e76-683c-4dde-b2fd-179b722b6ae5\") "
Jul 31 12:25:38.180002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.179836    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util\") pod \"914af701-5f79-4e15-b8d2-7f563c6a28d0\" (UID: \"914af701-5f79-4e15-b8d2-7f563c6a28d0\") "
Jul 31 12:25:38.180452 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.180417    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle" (OuterVolumeSpecName: "bundle") pod "914af701-5f79-4e15-b8d2-7f563c6a28d0" (UID: "914af701-5f79-4e15-b8d2-7f563c6a28d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:38.180612 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.180582    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle" (OuterVolumeSpecName: "bundle") pod "5b0d1e76-683c-4dde-b2fd-179b722b6ae5" (UID: "5b0d1e76-683c-4dde-b2fd-179b722b6ae5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:38.181965 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.181935    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c" (OuterVolumeSpecName: "kube-api-access-vvb9c") pod "5b0d1e76-683c-4dde-b2fd-179b722b6ae5" (UID: "5b0d1e76-683c-4dde-b2fd-179b722b6ae5"). InnerVolumeSpecName "kube-api-access-vvb9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:25:38.182063 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.181968    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt" (OuterVolumeSpecName: "kube-api-access-8cmqt") pod "914af701-5f79-4e15-b8d2-7f563c6a28d0" (UID: "914af701-5f79-4e15-b8d2-7f563c6a28d0"). InnerVolumeSpecName "kube-api-access-8cmqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:25:38.185163 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.185142    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util" (OuterVolumeSpecName: "util") pod "5b0d1e76-683c-4dde-b2fd-179b722b6ae5" (UID: "5b0d1e76-683c-4dde-b2fd-179b722b6ae5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:38.185624 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.185606    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util" (OuterVolumeSpecName: "util") pod "914af701-5f79-4e15-b8d2-7f563c6a28d0" (UID: "914af701-5f79-4e15-b8d2-7f563c6a28d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:38.281343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281317    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.281343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281339    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.281343 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281347    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.281553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281355    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/914af701-5f79-4e15-b8d2-7f563c6a28d0-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.281553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281364    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cmqt\" (UniqueName: \"kubernetes.io/projected/914af701-5f79-4e15-b8d2-7f563c6a28d0-kube-api-access-8cmqt\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.281553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.281374    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvb9c\" (UniqueName: \"kubernetes.io/projected/5b0d1e76-683c-4dde-b2fd-179b722b6ae5-kube-api-access-vvb9c\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:38.950517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.950428    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l"
Jul 31 12:25:38.950926 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.950434    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbvvj9l" event={"ID":"914af701-5f79-4e15-b8d2-7f563c6a28d0","Type":"ContainerDied","Data":"bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0"}
Jul 31 12:25:38.950926 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.950578    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdc140081ecaf9ecabde1ae9c894f22b9c289e2c609d695158e6bd1a129c5cf0"
Jul 31 12:25:38.952378 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.952358    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk"
Jul 31 12:25:38.952493 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.952391    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537n7nk" event={"ID":"5b0d1e76-683c-4dde-b2fd-179b722b6ae5","Type":"ContainerDied","Data":"57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5"}
Jul 31 12:25:38.952493 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:38.952427    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57835659cbac32474ab1af12ff1d0ddde8d09dbbff40989aa84795273a70f8e5"
Jul 31 12:25:39.080949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.080923    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:39.188888 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.188852    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd4ms\" (UniqueName: \"kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms\") pod \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") "
Jul 31 12:25:39.189083 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.188985    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle\") pod \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") "
Jul 31 12:25:39.189083 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.189021    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util\") pod \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\" (UID: \"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263\") "
Jul 31 12:25:39.189539 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.189509    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle" (OuterVolumeSpecName: "bundle") pod "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" (UID: "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:39.190975 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.190951    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms" (OuterVolumeSpecName: "kube-api-access-qd4ms") pod "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" (UID: "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263"). InnerVolumeSpecName "kube-api-access-qd4ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:25:39.196879 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.196841    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util" (OuterVolumeSpecName: "util") pod "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" (UID: "05e8a9a9-bb68-4ecd-a2f7-e40062b8f263"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:25:39.290247 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.290216    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:39.290247 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.290242    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:39.290247 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.290251    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd4ms\" (UniqueName: \"kubernetes.io/projected/05e8a9a9-bb68-4ecd-a2f7-e40062b8f263-kube-api-access-qd4ms\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:39.958930 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.958903    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs"
Jul 31 12:25:39.958930 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.958913    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183clwqvs" event={"ID":"05e8a9a9-bb68-4ecd-a2f7-e40062b8f263","Type":"ContainerDied","Data":"198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d"}
Jul 31 12:25:39.959333 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:39.958958    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="198a6ce7132bb5f9835a8f2015f32c8cfea2af9bd27f4399e9b0855156c2d29d"
Jul 31 12:25:50.925921 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:50.925878    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f7dcb6dd7-h72hl" podUID="371a2595-c1fe-4086-a6e2-341fb5030832" containerName="console" containerID="cri-o://4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c" gracePeriod=15
Jul 31 12:25:51.173267 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.173231    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:25:51.294044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294019    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294222 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294050    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294222 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294083    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294222 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294115    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294222 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294159    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294222 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294202    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdm6z\" (UniqueName: \"kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294499 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294226    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle\") pod \"371a2595-c1fe-4086-a6e2-341fb5030832\" (UID: \"371a2595-c1fe-4086-a6e2-341fb5030832\") "
Jul 31 12:25:51.294614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294555    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca" (OuterVolumeSpecName: "service-ca") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:25:51.294729 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294667    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config" (OuterVolumeSpecName: "console-config") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:25:51.294785 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294727    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:25:51.294785 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.294750    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:25:51.296212 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.296166    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:25:51.296212 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.296200    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:25:51.296406 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.296387    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z" (OuterVolumeSpecName: "kube-api-access-pdm6z") pod "371a2595-c1fe-4086-a6e2-341fb5030832" (UID: "371a2595-c1fe-4086-a6e2-341fb5030832"). InnerVolumeSpecName "kube-api-access-pdm6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:25:51.395556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395526    2532 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395551    2532 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395556 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395561    2532 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/371a2595-c1fe-4086-a6e2-341fb5030832-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395756 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395570    2532 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395756 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395578    2532 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395756 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395586    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdm6z\" (UniqueName: \"kubernetes.io/projected/371a2595-c1fe-4086-a6e2-341fb5030832-kube-api-access-pdm6z\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:51.395756 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:51.395596    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/371a2595-c1fe-4086-a6e2-341fb5030832-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:25:52.023592 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.023559    2532 generic.go:356] "Generic (PLEG): container finished" podID="371a2595-c1fe-4086-a6e2-341fb5030832" containerID="4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c" exitCode=0
Jul 31 12:25:52.024014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.023603    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcb6dd7-h72hl"
Jul 31 12:25:52.024014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.023646    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcb6dd7-h72hl" event={"ID":"371a2595-c1fe-4086-a6e2-341fb5030832","Type":"ContainerDied","Data":"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"}
Jul 31 12:25:52.024014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.023694    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcb6dd7-h72hl" event={"ID":"371a2595-c1fe-4086-a6e2-341fb5030832","Type":"ContainerDied","Data":"eee20a0643c3645c7352ab19020e02b17a2b4043aaa15908802e204f795d1fca"}
Jul 31 12:25:52.024014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.023714    2532 scope.go:117] "RemoveContainer" containerID="4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"
Jul 31 12:25:52.032373 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.032355    2532 scope.go:117] "RemoveContainer" containerID="4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"
Jul 31 12:25:52.032609 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:25:52.032593    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c\": container with ID starting with 4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c not found: ID does not exist" containerID="4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"
Jul 31 12:25:52.032655 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.032616    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c"} err="failed to get container status \"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c\": rpc error: code = NotFound desc = could not find container \"4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c\": container with ID starting with 4433e517ab6b823a2ffb2ab2cfe19752916ec9753480068320ed78bed702ee2c not found: ID does not exist"
Jul 31 12:25:52.051880 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.051857    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:25:52.063689 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:52.063668    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7dcb6dd7-h72hl"]
Jul 31 12:25:53.346146 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:25:53.346110    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="371a2595-c1fe-4086-a6e2-341fb5030832" path="/var/lib/kubelet/pods/371a2595-c1fe-4086-a6e2-341fb5030832/volumes"
Jul 31 12:26:08.784412 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784376    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"]
Jul 31 12:26:08.784776 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784763    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="extract"
Jul 31 12:26:08.784819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784778    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="extract"
Jul 31 12:26:08.784819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784786    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="util"
Jul 31 12:26:08.784819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784792    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="util"
Jul 31 12:26:08.784819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784813    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="util"
Jul 31 12:26:08.784819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784820    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="util"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784827    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784832    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784841    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784846    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784851    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784857    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784866    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784870    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784884    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784892    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784900    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="util"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784904    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="util"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784908    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784913    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="pull"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784919    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784923    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="extract"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784932    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="371a2595-c1fe-4086-a6e2-341fb5030832" containerName="console"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784937    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="371a2595-c1fe-4086-a6e2-341fb5030832" containerName="console"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784943    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="util"
Jul 31 12:26:08.784968 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.784947    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="util"
Jul 31 12:26:08.785590 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.785006    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="914af701-5f79-4e15-b8d2-7f563c6a28d0" containerName="extract"
Jul 31 12:26:08.785590 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.785017    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e8a9a9-bb68-4ecd-a2f7-e40062b8f263" containerName="extract"
Jul 31 12:26:08.785590 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.785023    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa4e150-affe-4788-b6d2-631e1da6ba08" containerName="extract"
Jul 31 12:26:08.785590 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.785028    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b0d1e76-683c-4dde-b2fd-179b722b6ae5" containerName="extract"
Jul 31 12:26:08.785590 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.785035    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="371a2595-c1fe-4086-a6e2-341fb5030832" containerName="console"
Jul 31 12:26:08.787261 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.787245    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:08.789684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.789655    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-54ldl\""
Jul 31 12:26:08.789834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.789709    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 31 12:26:08.789834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.789781    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 31 12:26:08.798107 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.798085    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"]
Jul 31 12:26:08.924688 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.924661    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"]
Jul 31 12:26:08.924899 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:08.924881    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-mzprr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm" podUID="cc20faf4-0af7-4667-9f02-f2d725b2be35"
Jul 31 12:26:08.950244 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.950222    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzprr\" (UniqueName: \"kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:08.950351 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.950254    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:08.974237 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.974209    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"]
Jul 31 12:26:08.976912 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:08.976897    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.014925 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.014903    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"]
Jul 31 12:26:09.051275 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.051218    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzprr\" (UniqueName: \"kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.051275 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.051260    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.051621 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.051603    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.067551 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.067530    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzprr\" (UniqueName: \"kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr\") pod \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.113455 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.113427    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.119552 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.119531    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:09.151714 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.151692    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brw4m\" (UniqueName: \"kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.151828 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.151742    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.252600 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252574    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume\") pod \"cc20faf4-0af7-4667-9f02-f2d725b2be35\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") "
Jul 31 12:26:09.252709 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252619    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzprr\" (UniqueName: \"kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr\") pod \"cc20faf4-0af7-4667-9f02-f2d725b2be35\" (UID: \"cc20faf4-0af7-4667-9f02-f2d725b2be35\") "
Jul 31 12:26:09.252768 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252752    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brw4m\" (UniqueName: \"kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.252807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252781    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cc20faf4-0af7-4667-9f02-f2d725b2be35" (UID: "cc20faf4-0af7-4667-9f02-f2d725b2be35"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:26:09.252854 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252840    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.252947 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.252927    2532 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc20faf4-0af7-4667-9f02-f2d725b2be35-extensions-socket-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:09.253350 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.253332    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.254850 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.254829    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr" (OuterVolumeSpecName: "kube-api-access-mzprr") pod "cc20faf4-0af7-4667-9f02-f2d725b2be35" (UID: "cc20faf4-0af7-4667-9f02-f2d725b2be35"). InnerVolumeSpecName "kube-api-access-mzprr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:26:09.275286 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.275266    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brw4m\" (UniqueName: \"kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.286114 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.286091    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:09.353474 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.353446    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzprr\" (UniqueName: \"kubernetes.io/projected/cc20faf4-0af7-4667-9f02-f2d725b2be35-kube-api-access-mzprr\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:09.424243 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.424206    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"]
Jul 31 12:26:09.429800 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:26:09.429753    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b72dd35_f81b_4e12_b3bc_1562b904110f.slice/crio-4cf852d0ca90f78da0925680fbe8bf005b67ef77e60f21aefb355070ec98ad6a WatchSource:0}: Error finding container 4cf852d0ca90f78da0925680fbe8bf005b67ef77e60f21aefb355070ec98ad6a: Status 404 returned error can't find the container with id 4cf852d0ca90f78da0925680fbe8bf005b67ef77e60f21aefb355070ec98ad6a
Jul 31 12:26:09.430688 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.430614    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"]
Jul 31 12:26:09.444923 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.444900    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"]
Jul 31 12:26:09.459128 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.459094    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:09.463550 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.463531    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.463685 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.463662    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"]
Jul 31 12:26:09.485712 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.485684    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"]
Jul 31 12:26:09.487917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.487892    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:09.655774 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.655699    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.655774 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.655746    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhrw\" (UniqueName: \"kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.756972 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.756945    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.757102 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.756985    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhrw\" (UniqueName: \"kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.757321 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.757304    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:09.774720 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:09.774695    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhrw\" (UniqueName: \"kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw\") pod \"kuadrant-operator-controller-manager-6bd956ccf-2mtzv\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:10.073870 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:10.073835    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:10.123683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:10.122938    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:10.128931 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:10.128891    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="cc20faf4-0af7-4667-9f02-f2d725b2be35" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:10.131084 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:10.131042    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-bjzbm\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="cc20faf4-0af7-4667-9f02-f2d725b2be35" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bjzbm"
Jul 31 12:26:10.245517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:10.245485    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:10.248494 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:26:10.248463    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc796bf_2a05_4735_af44_4e47eceefbdd.slice/crio-5dbb5ea0849ccfca95b57cfa1aba7392e17dc1c0dc7875795ba095d7b164bba1 WatchSource:0}: Error finding container 5dbb5ea0849ccfca95b57cfa1aba7392e17dc1c0dc7875795ba095d7b164bba1: Status 404 returned error can't find the container with id 5dbb5ea0849ccfca95b57cfa1aba7392e17dc1c0dc7875795ba095d7b164bba1
Jul 31 12:26:11.139838 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:11.139802    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" event={"ID":"acc796bf-2a05-4735-af44-4e47eceefbdd","Type":"ContainerStarted","Data":"5dbb5ea0849ccfca95b57cfa1aba7392e17dc1c0dc7875795ba095d7b164bba1"}
Jul 31 12:26:11.355669 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:11.355435    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc20faf4-0af7-4667-9f02-f2d725b2be35" path="/var/lib/kubelet/pods/cc20faf4-0af7-4667-9f02-f2d725b2be35/volumes"
Jul 31 12:26:14.162274 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.162218    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" containerName="manager" containerID="cri-o://0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785" gracePeriod=2
Jul 31 12:26:14.163915 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.163880    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" event={"ID":"acc796bf-2a05-4735-af44-4e47eceefbdd","Type":"ContainerStarted","Data":"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"}
Jul 31 12:26:14.164031 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.163956    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:14.164310 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:14.164282    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:14.184241 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:14.184212    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:14.184540 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.184504    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" podStartSLOduration=1.89641899 podStartE2EDuration="5.184490064s" totalImagesPullingTime="3.288071074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:26:14.182151112 +0000 UTC m=+661.414015861" watchObservedRunningTime="2026-07-31 12:26:14.184490064 +0000 UTC m=+661.416354813"
Jul 31 12:26:14.408904 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.408879    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:14.411204 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:14.411159    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:14.505739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.505710    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume\") pod \"7b72dd35-f81b-4e12-b3bc-1562b904110f\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") "
Jul 31 12:26:14.505926 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.505753    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brw4m\" (UniqueName: \"kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m\") pod \"7b72dd35-f81b-4e12-b3bc-1562b904110f\" (UID: \"7b72dd35-f81b-4e12-b3bc-1562b904110f\") "
Jul 31 12:26:14.506047 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.506026    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b72dd35-f81b-4e12-b3bc-1562b904110f" (UID: "7b72dd35-f81b-4e12-b3bc-1562b904110f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:26:14.507882 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.507860    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m" (OuterVolumeSpecName: "kube-api-access-brw4m") pod "7b72dd35-f81b-4e12-b3bc-1562b904110f" (UID: "7b72dd35-f81b-4e12-b3bc-1562b904110f"). InnerVolumeSpecName "kube-api-access-brw4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:26:14.606270 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.606240    2532 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b72dd35-f81b-4e12-b3bc-1562b904110f-extensions-socket-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:14.606270 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:14.606265    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brw4m\" (UniqueName: \"kubernetes.io/projected/7b72dd35-f81b-4e12-b3bc-1562b904110f-kube-api-access-brw4m\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:15.169970 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.169931    2532 generic.go:356] "Generic (PLEG): container finished" podID="7b72dd35-f81b-4e12-b3bc-1562b904110f" containerID="0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785" exitCode=2
Jul 31 12:26:15.170396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.169978    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:15.170396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.170022    2532 scope.go:117] "RemoveContainer" containerID="0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785"
Jul 31 12:26:15.172122 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:15.172093    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:15.179582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.179566    2532 scope.go:117] "RemoveContainer" containerID="0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785"
Jul 31 12:26:15.179847 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:15.179827    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785\": container with ID starting with 0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785 not found: ID does not exist" containerID="0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785"
Jul 31 12:26:15.179918 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.179853    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785"} err="failed to get container status \"0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785\": rpc error: code = NotFound desc = could not find container \"0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785\": container with ID starting with 0a3e430d10044e90e0b71f7b509048cb4068db302285334596613b2d672ad785 not found: ID does not exist"
Jul 31 12:26:15.180388 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:15.180368    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5r769\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5r769"
Jul 31 12:26:15.346080 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:15.346047    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" path="/var/lib/kubelet/pods/7b72dd35-f81b-4e12-b3bc-1562b904110f/volumes"
Jul 31 12:26:25.172524 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.172490    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:25.517831 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.517793    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:26:25.518231 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.518216    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" containerName="manager"
Jul 31 12:26:25.518293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.518232    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" containerName="manager"
Jul 31 12:26:25.518293 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.518285    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b72dd35-f81b-4e12-b3bc-1562b904110f" containerName="manager"
Jul 31 12:26:25.522517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.522498    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.533346 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.533322    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:26:25.594561 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.594530    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk79s\" (UniqueName: \"kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.594707 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.594565    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.695106 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.695074    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk79s\" (UniqueName: \"kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.695295 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.695114    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.695500 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.695482    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.704727 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.704705    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk79s\" (UniqueName: \"kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s\") pod \"kuadrant-operator-controller-manager-68cf589589-xqhm4\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.833548 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.833472    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:25.967751 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:25.967727    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:26:25.970648 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:26:25.970624    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4258006_60b9_4f3f_87e2_6c4943f14869.slice/crio-cae5c72ee3dfe1af4ecf9d3bc8cb1922c729ebe2ad092a38ab67ac1db41ab0ae WatchSource:0}: Error finding container cae5c72ee3dfe1af4ecf9d3bc8cb1922c729ebe2ad092a38ab67ac1db41ab0ae: Status 404 returned error can't find the container with id cae5c72ee3dfe1af4ecf9d3bc8cb1922c729ebe2ad092a38ab67ac1db41ab0ae
Jul 31 12:26:26.239340 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:26.239309    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" event={"ID":"b4258006-60b9-4f3f-87e2-6c4943f14869","Type":"ContainerStarted","Data":"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"}
Jul 31 12:26:26.239689 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:26.239382    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" event={"ID":"b4258006-60b9-4f3f-87e2-6c4943f14869","Type":"ContainerStarted","Data":"cae5c72ee3dfe1af4ecf9d3bc8cb1922c729ebe2ad092a38ab67ac1db41ab0ae"}
Jul 31 12:26:26.239689 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:26.239419    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:26.261542 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:26.261502    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" podStartSLOduration=1.261489277 podStartE2EDuration="1.261489277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:26:26.25821571 +0000 UTC m=+673.490080461" watchObservedRunningTime="2026-07-31 12:26:26.261489277 +0000 UTC m=+673.493354070"
Jul 31 12:26:37.247729 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.247697    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:26:37.302256 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.302222    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:37.302504 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.302457    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" podUID="acc796bf-2a05-4735-af44-4e47eceefbdd" containerName="manager" containerID="cri-o://56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95" gracePeriod=10
Jul 31 12:26:37.548461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.548434    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:37.594438 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.594410    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlhrw\" (UniqueName: \"kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw\") pod \"acc796bf-2a05-4735-af44-4e47eceefbdd\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") "
Jul 31 12:26:37.594586 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.594507    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume\") pod \"acc796bf-2a05-4735-af44-4e47eceefbdd\" (UID: \"acc796bf-2a05-4735-af44-4e47eceefbdd\") "
Jul 31 12:26:37.594983 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.594962    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "acc796bf-2a05-4735-af44-4e47eceefbdd" (UID: "acc796bf-2a05-4735-af44-4e47eceefbdd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:26:37.596361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.596339    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw" (OuterVolumeSpecName: "kube-api-access-tlhrw") pod "acc796bf-2a05-4735-af44-4e47eceefbdd" (UID: "acc796bf-2a05-4735-af44-4e47eceefbdd"). InnerVolumeSpecName "kube-api-access-tlhrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:26:37.695056 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.695020    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlhrw\" (UniqueName: \"kubernetes.io/projected/acc796bf-2a05-4735-af44-4e47eceefbdd-kube-api-access-tlhrw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:37.695056 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:37.695049    2532 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acc796bf-2a05-4735-af44-4e47eceefbdd-extensions-socket-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:26:38.312763 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.312718    2532 generic.go:356] "Generic (PLEG): container finished" podID="acc796bf-2a05-4735-af44-4e47eceefbdd" containerID="56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95" exitCode=0
Jul 31 12:26:38.313283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.312798    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"
Jul 31 12:26:38.313283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.312815    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" event={"ID":"acc796bf-2a05-4735-af44-4e47eceefbdd","Type":"ContainerDied","Data":"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"}
Jul 31 12:26:38.313283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.312969    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv" event={"ID":"acc796bf-2a05-4735-af44-4e47eceefbdd","Type":"ContainerDied","Data":"5dbb5ea0849ccfca95b57cfa1aba7392e17dc1c0dc7875795ba095d7b164bba1"}
Jul 31 12:26:38.313283 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.313005    2532 scope.go:117] "RemoveContainer" containerID="56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"
Jul 31 12:26:38.322116 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.322098    2532 scope.go:117] "RemoveContainer" containerID="56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"
Jul 31 12:26:38.322400 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:26:38.322383    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95\": container with ID starting with 56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95 not found: ID does not exist" containerID="56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"
Jul 31 12:26:38.322480 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.322406    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95"} err="failed to get container status \"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95\": rpc error: code = NotFound desc = could not find container \"56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95\": container with ID starting with 56d507ede091fe454dc1961ab0a0e728a5959aa0e578f90e6437469b1d22df95 not found: ID does not exist"
Jul 31 12:26:38.335973 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.335949    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:38.339367 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:38.339345    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-2mtzv"]
Jul 31 12:26:39.345379 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:39.345348    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acc796bf-2a05-4735-af44-4e47eceefbdd" path="/var/lib/kubelet/pods/acc796bf-2a05-4735-af44-4e47eceefbdd/volumes"
Jul 31 12:26:53.351254 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.351219    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:26:53.351857 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.351836    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc796bf-2a05-4735-af44-4e47eceefbdd" containerName="manager"
Jul 31 12:26:53.351910 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.351860    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc796bf-2a05-4735-af44-4e47eceefbdd" containerName="manager"
Jul 31 12:26:53.351980 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.351968    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="acc796bf-2a05-4735-af44-4e47eceefbdd" containerName="manager"
Jul 31 12:26:53.356432 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.356413    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.358679 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.358655    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 31 12:26:53.358785 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.358712    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gpmcc\""
Jul 31 12:26:53.363646 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.363620    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:26:53.388014 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.387987    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:26:53.431665 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.431630    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqfk5\" (UniqueName: \"kubernetes.io/projected/c601995e-a55e-431a-a3c6-6ab45a014143-kube-api-access-gqfk5\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.431805 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.431689    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c601995e-a55e-431a-a3c6-6ab45a014143-config-file\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.532624 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.532588    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c601995e-a55e-431a-a3c6-6ab45a014143-config-file\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.532780 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.532676    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqfk5\" (UniqueName: \"kubernetes.io/projected/c601995e-a55e-431a-a3c6-6ab45a014143-kube-api-access-gqfk5\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.533247 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.533228    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c601995e-a55e-431a-a3c6-6ab45a014143-config-file\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.540958 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.540934    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqfk5\" (UniqueName: \"kubernetes.io/projected/c601995e-a55e-431a-a3c6-6ab45a014143-kube-api-access-gqfk5\") pod \"limitador-limitador-85d9db4d5d-n4qdf\" (UID: \"c601995e-a55e-431a-a3c6-6ab45a014143\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.669341 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.669261    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:53.804910 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:53.804885    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:26:53.806836 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:26:53.806804    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc601995e_a55e_431a_a3c6_6ab45a014143.slice/crio-b28182174e313ec6b948a064326745b9c8ec1bd7f8e52733ca6da4b33ea36074 WatchSource:0}: Error finding container b28182174e313ec6b948a064326745b9c8ec1bd7f8e52733ca6da4b33ea36074: Status 404 returned error can't find the container with id b28182174e313ec6b948a064326745b9c8ec1bd7f8e52733ca6da4b33ea36074
Jul 31 12:26:54.401479 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:54.401438    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf" event={"ID":"c601995e-a55e-431a-a3c6-6ab45a014143","Type":"ContainerStarted","Data":"b28182174e313ec6b948a064326745b9c8ec1bd7f8e52733ca6da4b33ea36074"}
Jul 31 12:26:56.413942 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:56.413905    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf" event={"ID":"c601995e-a55e-431a-a3c6-6ab45a014143","Type":"ContainerStarted","Data":"0cbb454acbb1f3a30dcad366c72d86cc121f893cd802521e52dc8a38f4cb28b0"}
Jul 31 12:26:56.414400 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:56.414004    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:26:56.431936 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:26:56.431886    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf" podStartSLOduration=1.744691069 podStartE2EDuration="3.431871883s" totalImagesPullingTime="1.687180814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:26:56.430280589 +0000 UTC m=+703.662145339" watchObservedRunningTime="2026-07-31 12:26:56.431871883 +0000 UTC m=+703.663736633"
Jul 31 12:27:07.420574 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:27:07.420500    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"
Jul 31 12:28:08.184288 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:08.184254    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:28:34.746287 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:34.746245    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:28:41.348851 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.348813    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"]
Jul 31 12:28:41.352498 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.352476    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.355933 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.355912    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 31 12:28:41.356158 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.355929    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 31 12:28:41.356158 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.355942    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 31 12:28:41.356158 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.355912    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vmtl2\""
Jul 31 12:28:41.360326 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.360305    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"]
Jul 31 12:28:41.471517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471482    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.471674 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471531    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.471674 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471621    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxj8m\" (UniqueName: \"kubernetes.io/projected/3fccc59d-9a0d-45a6-9138-b71479d27aab-kube-api-access-fxj8m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.471674 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471656    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.471850 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471767    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.471850 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.471809    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccc59d-9a0d-45a6-9138-b71479d27aab-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572443 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572408    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572452    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572500    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxj8m\" (UniqueName: \"kubernetes.io/projected/3fccc59d-9a0d-45a6-9138-b71479d27aab-kube-api-access-fxj8m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572521    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572559    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572604 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572595    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccc59d-9a0d-45a6-9138-b71479d27aab-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572875 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572846    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.572982 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.572919    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.573023 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.573001    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.574728 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.574710    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fccc59d-9a0d-45a6-9138-b71479d27aab-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.574988 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.574972    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccc59d-9a0d-45a6-9138-b71479d27aab-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.579853 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.579834    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxj8m\" (UniqueName: \"kubernetes.io/projected/3fccc59d-9a0d-45a6-9138-b71479d27aab-kube-api-access-fxj8m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnxnt\" (UID: \"3fccc59d-9a0d-45a6-9138-b71479d27aab\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.664460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.664387    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:41.799210 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.799180    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"]
Jul 31 12:28:41.799944 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:28:41.799920    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fccc59d_9a0d_45a6_9138_b71479d27aab.slice/crio-c74ba23ee3fb68cb6fa625491d561cf16cbd8fcd9ed0e899acc3618f09c9f5d9 WatchSource:0}: Error finding container c74ba23ee3fb68cb6fa625491d561cf16cbd8fcd9ed0e899acc3618f09c9f5d9: Status 404 returned error can't find the container with id c74ba23ee3fb68cb6fa625491d561cf16cbd8fcd9ed0e899acc3618f09c9f5d9
Jul 31 12:28:41.951057 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.950968    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:28:41.981158 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:41.981125    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt" event={"ID":"3fccc59d-9a0d-45a6-9138-b71479d27aab","Type":"ContainerStarted","Data":"c74ba23ee3fb68cb6fa625491d561cf16cbd8fcd9ed0e899acc3618f09c9f5d9"}
Jul 31 12:28:42.988606 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:42.988573    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt" event={"ID":"3fccc59d-9a0d-45a6-9138-b71479d27aab","Type":"ContainerStarted","Data":"e40fdaf226e0676a58da3124bfc8a41dfd5a289cdbf7f6fd7d1a1bb41c14505d"}
Jul 31 12:28:43.009549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:43.009501    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt" podStartSLOduration=0.996053084 podStartE2EDuration="2.009486466s" totalImagesPullingTime="1.013433382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:28:43.00684631 +0000 UTC m=+810.238711084" watchObservedRunningTime="2026-07-31 12:28:43.009486466 +0000 UTC m=+810.241351217"
Jul 31 12:28:51.664882 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:51.664849    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:51.665278 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:51.664925    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:51.677426 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:51.677400    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:51.855975 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:51.855939    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:28:52.048950 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:52.048923    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnxnt"
Jul 31 12:28:58.355450 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.355407    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"]
Jul 31 12:28:58.359081 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.359056    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.362096 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.362075    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 31 12:28:58.371024 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.371000    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"]
Jul 31 12:28:58.419431 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419406    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.419549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419462    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b73da28-0877-4d16-b8f9-1fb21b361a85-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.419549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419542    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.419628 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419565    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.419628 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419583    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lcwn\" (UniqueName: \"kubernetes.io/projected/7b73da28-0877-4d16-b8f9-1fb21b361a85-kube-api-access-9lcwn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.419628 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.419614    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520811 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520782    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520825    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520862    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b73da28-0877-4d16-b8f9-1fb21b361a85-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520920    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520939    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.520971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.520956    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lcwn\" (UniqueName: \"kubernetes.io/projected/7b73da28-0877-4d16-b8f9-1fb21b361a85-kube-api-access-9lcwn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.521391 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.521339    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.521391 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.521377    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.521562 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.521423    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.523094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.523078    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b73da28-0877-4d16-b8f9-1fb21b361a85-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.523405 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.523384    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b73da28-0877-4d16-b8f9-1fb21b361a85-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.529125 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.529103    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lcwn\" (UniqueName: \"kubernetes.io/projected/7b73da28-0877-4d16-b8f9-1fb21b361a85-kube-api-access-9lcwn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc\" (UID: \"7b73da28-0877-4d16-b8f9-1fb21b361a85\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.670784 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.670710    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:28:58.810433 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:58.810405    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"]
Jul 31 12:28:58.812512 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:28:58.812481    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b73da28_0877_4d16_b8f9_1fb21b361a85.slice/crio-db2491fb2f698fd9a849727642a183212ea6bd6e7cfff99bf0293b831eb15c6a WatchSource:0}: Error finding container db2491fb2f698fd9a849727642a183212ea6bd6e7cfff99bf0293b831eb15c6a: Status 404 returned error can't find the container with id db2491fb2f698fd9a849727642a183212ea6bd6e7cfff99bf0293b831eb15c6a
Jul 31 12:28:59.079627 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:59.079593    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc" event={"ID":"7b73da28-0877-4d16-b8f9-1fb21b361a85","Type":"ContainerStarted","Data":"c1b6fe74f821324a482bf0801e6ed1f25512aa7bbc433e30255890262dba16a3"}
Jul 31 12:28:59.079774 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:59.079649    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc" event={"ID":"7b73da28-0877-4d16-b8f9-1fb21b361a85","Type":"ContainerStarted","Data":"db2491fb2f698fd9a849727642a183212ea6bd6e7cfff99bf0293b831eb15c6a"}
Jul 31 12:28:59.103116 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:59.103057    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc" podStartSLOduration=0.926946758 podStartE2EDuration="1.103036799s" totalImagesPullingTime="176.090041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:28:59.09995411 +0000 UTC m=+826.331818884" watchObservedRunningTime="2026-07-31 12:28:59.103036799 +0000 UTC m=+826.334901550"
Jul 31 12:28:59.564430 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:28:59.564394    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:29:04.750120 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:04.750078    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:29:08.671027 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:08.670979    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:29:08.671475 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:08.671057    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:29:08.683363 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:08.683343    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:29:09.145363 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:09.145329    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc"
Jul 31 12:29:10.851958 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.851920    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"]
Jul 31 12:29:10.855578 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.855557    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.857893 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.857866    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 31 12:29:10.865206 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.865181    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"]
Jul 31 12:29:10.931335 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931299    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jpd4\" (UniqueName: \"kubernetes.io/projected/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-kube-api-access-5jpd4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.931335 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931331    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.931587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931366    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.931587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931427    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.931587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931449    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:10.931587 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:10.931469    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032144 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032108    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032144 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032146    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032165    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032246    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jpd4\" (UniqueName: \"kubernetes.io/projected/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-kube-api-access-5jpd4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032263    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032304    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032685 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032663    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032746 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032693    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.032801 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.032761    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.034479 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.034455    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.034724 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.034709    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.039595 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.039570    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jpd4\" (UniqueName: \"kubernetes.io/projected/8717eb71-9a2b-4d82-8475-ec0cd0a891c3-kube-api-access-5jpd4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g85tf\" (UID: \"8717eb71-9a2b-4d82-8475-ec0cd0a891c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.166793 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.166717    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:11.299690 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:11.299622    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"]
Jul 31 12:29:11.302475 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:29:11.302440    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8717eb71_9a2b_4d82_8475_ec0cd0a891c3.slice/crio-d31a617b67904058f57292372fde30571f9a4190b9367e304200768f29f0c37c WatchSource:0}: Error finding container d31a617b67904058f57292372fde30571f9a4190b9367e304200768f29f0c37c: Status 404 returned error can't find the container with id d31a617b67904058f57292372fde30571f9a4190b9367e304200768f29f0c37c
Jul 31 12:29:12.152978 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:12.152945    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:29:12.161047 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:12.161019    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf" event={"ID":"8717eb71-9a2b-4d82-8475-ec0cd0a891c3","Type":"ContainerStarted","Data":"6e953fb4a38fdb3f84a29974ba161d0892e8bb78aaf35eaf36ad0af91a2e6aa2"}
Jul 31 12:29:12.161192 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:12.161056    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf" event={"ID":"8717eb71-9a2b-4d82-8475-ec0cd0a891c3","Type":"ContainerStarted","Data":"d31a617b67904058f57292372fde30571f9a4190b9367e304200768f29f0c37c"}
Jul 31 12:29:12.177949 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:12.177905    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf" podStartSLOduration=1.994974264 podStartE2EDuration="2.177890347s" totalImagesPullingTime="182.916083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:29:12.17578963 +0000 UTC m=+839.407654381" watchObservedRunningTime="2026-07-31 12:29:12.177890347 +0000 UTC m=+839.409755097"
Jul 31 12:29:21.167149 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:21.167117    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:21.167149 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:21.167154    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:21.179467 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:21.179438    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:29:21.221837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:29:21.221808    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g85tf"
Jul 31 12:30:54.551392 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:30:54.551360    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:31:04.467070 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:31:04.467034    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:31:16.054598 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:31:16.054560    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:32:56.263907 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:32:56.263868    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:33:05.550363 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:33:05.550269    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:05.752440 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:05.752402    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:12.549738 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:12.549700    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:28.447822 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:28.447785    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:34.248301 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:34.248265    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:43.459320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:43.459244    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:34:50.357259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:34:50.357220    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:35:04.180686 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:35:04.180657    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:35:08.956442 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:35:08.956411    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:35:16.572126 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:35:16.572086    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:35:36.157151 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:35:36.157115    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:35:46.858201 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:35:46.858151    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:09.452819 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:09.452736    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:19.952498 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:19.952455    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:26.452145 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:26.452110    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:33.458361 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:33.458329    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:40.554255 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:40.554218    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:46.352833 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:46.352794    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:51.255979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:51.255942    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:36:59.142774 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:36:59.142736    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:12.754937 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:12.754899    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:17.850900 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:17.850861    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:28.562142 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:28.562107    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:33.253777 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:33.253741    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:39.851561 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:39.851481    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:48.744223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:48.744182    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:37:54.148192 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:37:54.148146    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:38:05.252560 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:38:05.252523    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:38:59.896506 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:38:59.896461    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:38:59.897047 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:38:59.896716    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" podUID="b4258006-60b9-4f3f-87e2-6c4943f14869" containerName="manager" containerID="cri-o://d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30" gracePeriod=10
Jul 31 12:39:00.145357 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.145333    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:39:00.258902 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.258871    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume\") pod \"b4258006-60b9-4f3f-87e2-6c4943f14869\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") "
Jul 31 12:39:00.259049 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.258915    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk79s\" (UniqueName: \"kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s\") pod \"b4258006-60b9-4f3f-87e2-6c4943f14869\" (UID: \"b4258006-60b9-4f3f-87e2-6c4943f14869\") "
Jul 31 12:39:00.259237 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.259215    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b4258006-60b9-4f3f-87e2-6c4943f14869" (UID: "b4258006-60b9-4f3f-87e2-6c4943f14869"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:39:00.260845 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.260825    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s" (OuterVolumeSpecName: "kube-api-access-dk79s") pod "b4258006-60b9-4f3f-87e2-6c4943f14869" (UID: "b4258006-60b9-4f3f-87e2-6c4943f14869"). InnerVolumeSpecName "kube-api-access-dk79s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:39:00.360215 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.360161    2532 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4258006-60b9-4f3f-87e2-6c4943f14869-extensions-socket-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:39:00.360215 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.360214    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk79s\" (UniqueName: \"kubernetes.io/projected/b4258006-60b9-4f3f-87e2-6c4943f14869-kube-api-access-dk79s\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:39:00.445880 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.445851    2532 generic.go:356] "Generic (PLEG): container finished" podID="b4258006-60b9-4f3f-87e2-6c4943f14869" containerID="d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30" exitCode=0
Jul 31 12:39:00.446006 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.445901    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" event={"ID":"b4258006-60b9-4f3f-87e2-6c4943f14869","Type":"ContainerDied","Data":"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"}
Jul 31 12:39:00.446006 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.445910    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"
Jul 31 12:39:00.446006 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.445944    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4" event={"ID":"b4258006-60b9-4f3f-87e2-6c4943f14869","Type":"ContainerDied","Data":"cae5c72ee3dfe1af4ecf9d3bc8cb1922c729ebe2ad092a38ab67ac1db41ab0ae"}
Jul 31 12:39:00.446006 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.445964    2532 scope.go:117] "RemoveContainer" containerID="d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"
Jul 31 12:39:00.459657 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.459639    2532 scope.go:117] "RemoveContainer" containerID="d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"
Jul 31 12:39:00.459917 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:39:00.459895    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30\": container with ID starting with d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30 not found: ID does not exist" containerID="d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"
Jul 31 12:39:00.459969 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.459925    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30"} err="failed to get container status \"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30\": rpc error: code = NotFound desc = could not find container \"d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30\": container with ID starting with d6ac387469a39b9901ec4a6e79f24f9c65bd7b3a603be8ae7bd9055b8afd1c30 not found: ID does not exist"
Jul 31 12:39:00.468046 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.468027    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:39:00.473735 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:00.473715    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xqhm4"]
Jul 31 12:39:01.347706 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:39:01.347671    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4258006-60b9-4f3f-87e2-6c4943f14869" path="/var/lib/kubelet/pods/b4258006-60b9-4f3f-87e2-6c4943f14869/volumes"
Jul 31 12:40:00.600052 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.600018    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:40:00.600633 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.600614    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4258006-60b9-4f3f-87e2-6c4943f14869" containerName="manager"
Jul 31 12:40:00.600711 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.600637    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4258006-60b9-4f3f-87e2-6c4943f14869" containerName="manager"
Jul 31 12:40:00.600770 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.600757    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4258006-60b9-4f3f-87e2-6c4943f14869" containerName="manager"
Jul 31 12:40:00.603789 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.603769    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.606355 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.606335    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-54ldl\""
Jul 31 12:40:00.613623 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.613599    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:40:00.788539 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.788506    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7wvv\" (UniqueName: \"kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.788539 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.788542    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.889998 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.889909    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7wvv\" (UniqueName: \"kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.889998 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.889962    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.890326 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.890304    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.898785 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.898757    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7wvv\" (UniqueName: \"kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv\") pod \"kuadrant-operator-controller-manager-68cf589589-84qdw\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:00.915592 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:00.915569    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:01.057240 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:01.057217    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:40:01.059759 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:40:01.059732    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39f0541_ce76_4e59_ba43_ed92bfd8ccd9.slice/crio-06d572bafdde79b2a814fc019d230dd3d4c2dde6291870695f7c6047574da2e0 WatchSource:0}: Error finding container 06d572bafdde79b2a814fc019d230dd3d4c2dde6291870695f7c6047574da2e0: Status 404 returned error can't find the container with id 06d572bafdde79b2a814fc019d230dd3d4c2dde6291870695f7c6047574da2e0
Jul 31 12:40:01.771108 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:01.771068    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" event={"ID":"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9","Type":"ContainerStarted","Data":"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"}
Jul 31 12:40:01.771493 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:01.771132    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" event={"ID":"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9","Type":"ContainerStarted","Data":"06d572bafdde79b2a814fc019d230dd3d4c2dde6291870695f7c6047574da2e0"}
Jul 31 12:40:01.771493 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:01.771158    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:01.792120 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:01.792074    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" podStartSLOduration=1.792058747 podStartE2EDuration="1.792058747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:40:01.790336644 +0000 UTC m=+1489.022201395" watchObservedRunningTime="2026-07-31 12:40:01.792058747 +0000 UTC m=+1489.023923497"
Jul 31 12:40:12.780304 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:12.780272    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:40:23.147719 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:23.147682    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:40:30.144111 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:30.144069    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:40:51.349189 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:40:51.349142    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:02.454519 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:02.454482    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:09.545999 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:09.545959    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:17.544351 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:17.544311    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:23.142835 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:23.142798    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:32.943538 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:32.943504    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:39.346912 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:39.346874    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:48.248822 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:48.248780    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:41:54.857332 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:41:54.857294    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:03.550115 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:03.550072    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:09.676447 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:09.676367    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:19.250755 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:19.250711    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:41.768347 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:41.768309    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:48.155762 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:48.155726    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:42:59.518534 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:42:59.518498    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:43:01.258358 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.258328    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:01.262143 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.262122    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.264702 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.264677    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6d50556f-kserve-self-signed-certs\""
Jul 31 12:43:01.276320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.276296    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:01.404161 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404135    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.404320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404206    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.404320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404233    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.404320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404249    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.404320 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404281    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjd4x\" (UniqueName: \"kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.404463 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.404391    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505525 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505494    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505680 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505535    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505680 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505573    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505680 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505593    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505680 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505608    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505886 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505758    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjd4x\" (UniqueName: \"kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.505938 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505915    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.506002 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.505941    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.506067 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.506048    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.507736 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.507708    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.508036 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.508019    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.514376 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.514324    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjd4x\" (UniqueName: \"kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x\") pod \"e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") " pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.571806 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.571782    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:01.708092 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.708061    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:01.710028 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:43:01.710002    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f60e9de_b91c_4ed1_8fd0_e20a43e2920b.slice/crio-d77c011a9a7f2f368923b5f2d0a6a0d2bc60fc496347081543b641f6b579829b WatchSource:0}: Error finding container d77c011a9a7f2f368923b5f2d0a6a0d2bc60fc496347081543b641f6b579829b: Status 404 returned error can't find the container with id d77c011a9a7f2f368923b5f2d0a6a0d2bc60fc496347081543b641f6b579829b
Jul 31 12:43:01.776482 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:01.776455    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" event={"ID":"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b","Type":"ContainerStarted","Data":"d77c011a9a7f2f368923b5f2d0a6a0d2bc60fc496347081543b641f6b579829b"}
Jul 31 12:43:02.782976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:02.782944    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" event={"ID":"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b","Type":"ContainerStarted","Data":"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"}
Jul 31 12:43:02.801479 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:02.801426    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" podStartSLOduration=1.801409515 podStartE2EDuration="1.801409515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:43:02.798849784 +0000 UTC m=+1670.030714538" watchObservedRunningTime="2026-07-31 12:43:02.801409515 +0000 UTC m=+1670.033274265"
Jul 31 12:43:08.444450 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:08.444412    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:43:11.572279 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:11.572240    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:11.572670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:11.572292    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:11.584250 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:11.584223    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:11.843964 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:11.843879    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:30.034538 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:30.034503    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:31.652925 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:31.652885    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:43:39.955431 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:39.955356    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:43:45.053573 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.053534    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" podUID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" containerName="main" containerID="cri-o://754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb" gracePeriod=45
Jul 31 12:43:45.256398 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.256359    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:43:45.297518 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.297496    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:45.384334 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384247    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384334 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384283    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384347    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384401    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384428    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjd4x\" (UniqueName: \"kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384557 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384450    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir\") pod \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\" (UID: \"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b\") "
Jul 31 12:43:45.384753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384569    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:43:45.384753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384628    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home" (OuterVolumeSpecName: "home") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:43:45.384753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384731    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:43:45.384905 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384858    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:45.384905 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384881    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:45.384905 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.384894    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:45.386585 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.386558    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:43:45.386701 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.386606    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x" (OuterVolumeSpecName: "kube-api-access-tjd4x") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "kube-api-access-tjd4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:43:45.386701 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.386641    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm" (OuterVolumeSpecName: "dshm") pod "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" (UID: "5f60e9de-b91c-4ed1-8fd0-e20a43e2920b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:43:45.486156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.486132    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:45.486156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.486155    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:45.486323 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:45.486164    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjd4x\" (UniqueName: \"kubernetes.io/projected/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b-kube-api-access-tjd4x\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:43:46.041181 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.041132    2532 generic.go:356] "Generic (PLEG): container finished" podID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" containerID="754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb" exitCode=0
Jul 31 12:43:46.041372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.041191    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"
Jul 31 12:43:46.041372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.041222    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" event={"ID":"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b","Type":"ContainerDied","Data":"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"}
Jul 31 12:43:46.041372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.041274    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7" event={"ID":"5f60e9de-b91c-4ed1-8fd0-e20a43e2920b","Type":"ContainerDied","Data":"d77c011a9a7f2f368923b5f2d0a6a0d2bc60fc496347081543b641f6b579829b"}
Jul 31 12:43:46.041372 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.041297    2532 scope.go:117] "RemoveContainer" containerID="754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"
Jul 31 12:43:46.050601 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.050584    2532 scope.go:117] "RemoveContainer" containerID="754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"
Jul 31 12:43:46.050888 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:43:46.050870    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb\": container with ID starting with 754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb not found: ID does not exist" containerID="754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"
Jul 31 12:43:46.050950 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.050895    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb"} err="failed to get container status \"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb\": rpc error: code = NotFound desc = could not find container \"754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb\": container with ID starting with 754f7b7a9e8b5af94a617a8b024b7e16d086b50ee5acb4f4020f0a34dda4bdfb not found: ID does not exist"
Jul 31 12:43:46.062561 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.062539    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:46.064836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:46.064816    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6d50556f-kserve-77c96767dc-jrrh7"]
Jul 31 12:43:47.346673 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:47.346636    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" path="/var/lib/kubelet/pods/5f60e9de-b91c-4ed1-8fd0-e20a43e2920b/volumes"
Jul 31 12:43:54.952135 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:43:54.952096    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:02.244779 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:02.244738    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:08.050536 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:08.050499    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:14.553241 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:14.553203    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:20.551386 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:20.551343    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:25.242524 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:25.242484    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:34.646682 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:34.646649    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:41.050100 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:41.050060    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:44:57.453491 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:44:57.453450    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:04.455362 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:04.455325    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:15.754593 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:15.754557    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:23.848048 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:23.848016    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:29.650251 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:29.650214    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:34.349365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:34.349330    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:40.151690 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:40.151651    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:48.855826 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:48.855795    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:45:54.750018 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:45:54.749983    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:06.055044 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:06.054999    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:12.399891 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:12.399854    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:23.147205 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:23.147161    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:38.548384 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:38.548308    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:44.546456 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:44.546418    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:46:50.552912 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:46:50.552878    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:47:01.752907 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:01.752868    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:47:10.749448 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:10.749406    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:47:15.952732 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:15.952697    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:47:31.159157 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:31.159119    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:47:52.809295 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.809260    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"]
Jul 31 12:47:52.809741 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.809651    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" containerName="main"
Jul 31 12:47:52.809741 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.809662    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" containerName="main"
Jul 31 12:47:52.809741 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.809724    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f60e9de-b91c-4ed1-8fd0-e20a43e2920b" containerName="main"
Jul 31 12:47:52.812635 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.812619    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:52.814999 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.814981    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 12:47:52.815108 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.815088    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 12:47:52.819120 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.819094    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"]
Jul 31 12:47:52.836886 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.836861    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk\" (UID: \"c6dd336b-f605-4e0f-8787-1c749fb3aa8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:52.938253 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.938215    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk\" (UID: \"c6dd336b-f605-4e0f-8787-1c749fb3aa8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:52.938837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:52.938817    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk\" (UID: \"c6dd336b-f605-4e0f-8787-1c749fb3aa8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:53.124005 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:53.123907    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:53.267414 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:53.267373    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"]
Jul 31 12:47:53.269718 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:47:53.269688    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6dd336b_f605_4e0f_8787_1c749fb3aa8e.slice/crio-458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03 WatchSource:0}: Error finding container 458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03: Status 404 returned error can't find the container with id 458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03
Jul 31 12:47:53.272138 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:53.272118    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 12:47:53.442908 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:53.442817    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk" event={"ID":"c6dd336b-f605-4e0f-8787-1c749fb3aa8e","Type":"ContainerStarted","Data":"458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03"}
Jul 31 12:47:56.463581 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:56.463542    2532 generic.go:356] "Generic (PLEG): container finished" podID="c6dd336b-f605-4e0f-8787-1c749fb3aa8e" containerID="5274245402c9328acb3f0fef1c161ee7776c551f5ef9a40e957611fbaef00f20" exitCode=0
Jul 31 12:47:56.463965 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:56.463641    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk" event={"ID":"c6dd336b-f605-4e0f-8787-1c749fb3aa8e","Type":"ContainerDied","Data":"5274245402c9328acb3f0fef1c161ee7776c551f5ef9a40e957611fbaef00f20"}
Jul 31 12:47:57.600186 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:57.600146    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:57.680692 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:57.680658    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca\") pod \"c6dd336b-f605-4e0f-8787-1c749fb3aa8e\" (UID: \"c6dd336b-f605-4e0f-8787-1c749fb3aa8e\") "
Jul 31 12:47:57.681010 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:57.680988    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6dd336b-f605-4e0f-8787-1c749fb3aa8e" (UID: "c6dd336b-f605-4e0f-8787-1c749fb3aa8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:47:57.782229 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:57.782142    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dd336b-f605-4e0f-8787-1c749fb3aa8e-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:47:58.475573 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:58.475542    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"
Jul 31 12:47:58.475753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:58.475572    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk" event={"ID":"c6dd336b-f605-4e0f-8787-1c749fb3aa8e","Type":"ContainerDied","Data":"458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03"}
Jul 31 12:47:58.475753 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:47:58.475615    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="458cfe1bdf522a96ba8ed7a470b907ddd81c21a27510e1293c0ef57f62decb03"
Jul 31 12:48:09.013506 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:09.013415    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"]
Jul 31 12:48:09.015553 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:09.015531    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84c24c8b-rdbwk"]
Jul 31 12:48:09.346009 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:09.345927    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6dd336b-f605-4e0f-8787-1c749fb3aa8e" path="/var/lib/kubelet/pods/c6dd336b-f605-4e0f-8787-1c749fb3aa8e/volumes"
Jul 31 12:48:11.653034 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:11.652997    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:48:19.450596 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:19.450557    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:48:38.024160 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.024122    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"]
Jul 31 12:48:38.024586 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.024543    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6dd336b-f605-4e0f-8787-1c749fb3aa8e" containerName="revoke-keys"
Jul 31 12:48:38.024586 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.024554    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6dd336b-f605-4e0f-8787-1c749fb3aa8e" containerName="revoke-keys"
Jul 31 12:48:38.024661 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.024621    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6dd336b-f605-4e0f-8787-1c749fb3aa8e" containerName="revoke-keys"
Jul 31 12:48:38.032428 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.032377    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:38.034923 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.034899    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"]
Jul 31 12:48:38.035046 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.034930    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 12:48:38.035046 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.034905    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 12:48:38.051461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.051440    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk\" (UID: \"528da54b-741a-4e80-a1a7-6e33f166b1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:38.152042 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.152001    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk\" (UID: \"528da54b-741a-4e80-a1a7-6e33f166b1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:38.152582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.152563    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk\" (UID: \"528da54b-741a-4e80-a1a7-6e33f166b1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:38.343628 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.343547    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:38.472412 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.472382    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"]
Jul 31 12:48:38.474163 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:48:38.474130    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528da54b_741a_4e80_a1a7_6e33f166b1ae.slice/crio-4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b WatchSource:0}: Error finding container 4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b: Status 404 returned error can't find the container with id 4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b
Jul 31 12:48:38.700485 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:38.700404    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk" event={"ID":"528da54b-741a-4e80-a1a7-6e33f166b1ae","Type":"ContainerStarted","Data":"4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b"}
Jul 31 12:48:39.706835 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:39.706802    2532 generic.go:356] "Generic (PLEG): container finished" podID="528da54b-741a-4e80-a1a7-6e33f166b1ae" containerID="5d9a17ae368a324cfaa70776e1aea14091fd7a0c5ec9e5c53f616ae03d9cb659" exitCode=0
Jul 31 12:48:39.707259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:39.706894    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk" event={"ID":"528da54b-741a-4e80-a1a7-6e33f166b1ae","Type":"ContainerDied","Data":"5d9a17ae368a324cfaa70776e1aea14091fd7a0c5ec9e5c53f616ae03d9cb659"}
Jul 31 12:48:40.839602 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:40.839579    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:40.876955 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:40.876931    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca\") pod \"528da54b-741a-4e80-a1a7-6e33f166b1ae\" (UID: \"528da54b-741a-4e80-a1a7-6e33f166b1ae\") "
Jul 31 12:48:40.877278 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:40.877256    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "528da54b-741a-4e80-a1a7-6e33f166b1ae" (UID: "528da54b-741a-4e80-a1a7-6e33f166b1ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:48:40.978396 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:40.978310    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/528da54b-741a-4e80-a1a7-6e33f166b1ae-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:48:41.719086 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:41.719055    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk" event={"ID":"528da54b-741a-4e80-a1a7-6e33f166b1ae","Type":"ContainerDied","Data":"4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b"}
Jul 31 12:48:41.719259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:41.719108    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4af8c7e619fbd9e47ab79b7670f8c00a3bb37a156a68cba10fe490ec8bc80b0b"
Jul 31 12:48:41.719259 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:41.719068    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"
Jul 31 12:48:44.453967 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:44.453930    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"]
Jul 31 12:48:44.456215 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:44.456190    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-643d0a88-8trzk"]
Jul 31 12:48:45.346542 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:45.346509    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="528da54b-741a-4e80-a1a7-6e33f166b1ae" path="/var/lib/kubelet/pods/528da54b-741a-4e80-a1a7-6e33f166b1ae/volumes"
Jul 31 12:48:47.162697 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:47.162660    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:48:55.266779 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:48:55.266740    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:49:13.466551 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.466515    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"]
Jul 31 12:49:13.466979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.466887    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528da54b-741a-4e80-a1a7-6e33f166b1ae" containerName="revoke-keys"
Jul 31 12:49:13.466979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.466900    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="528da54b-741a-4e80-a1a7-6e33f166b1ae" containerName="revoke-keys"
Jul 31 12:49:13.466979 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.466977    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="528da54b-741a-4e80-a1a7-6e33f166b1ae" containerName="revoke-keys"
Jul 31 12:49:13.470181 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.470155    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:13.472563 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.472536    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 12:49:13.472683 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.472597    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 12:49:13.477529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.477506    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"]
Jul 31 12:49:13.573837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.573805    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b\" (UID: \"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:13.674543 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.674516    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b\" (UID: \"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:13.675062 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.675044    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b\" (UID: \"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:13.781836 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.781776    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:13.914164 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:13.914128    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"]
Jul 31 12:49:13.915586 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:49:13.915558    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ac57bd_2ffa_4829_8f9a_7b7dd20e5862.slice/crio-324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37 WatchSource:0}: Error finding container 324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37: Status 404 returned error can't find the container with id 324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37
Jul 31 12:49:14.912620 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:14.912532    2532 generic.go:356] "Generic (PLEG): container finished" podID="e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" containerID="919bea6bd286899651209d07e219bb321a1d81dafc8a73f8efe6d64f19c7de81" exitCode=0
Jul 31 12:49:14.912971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:14.912621    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b" event={"ID":"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862","Type":"ContainerDied","Data":"919bea6bd286899651209d07e219bb321a1d81dafc8a73f8efe6d64f19c7de81"}
Jul 31 12:49:14.912971 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:14.912679    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b" event={"ID":"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862","Type":"ContainerStarted","Data":"324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37"}
Jul 31 12:49:16.046315 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.046287    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:16.196625 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.196553    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca\") pod \"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862\" (UID: \"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862\") "
Jul 31 12:49:16.196877 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.196856    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" (UID: "e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:49:16.297998 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.297971    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:49:16.924397 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.924363    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"
Jul 31 12:49:16.924588 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.924360    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b" event={"ID":"e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862","Type":"ContainerDied","Data":"324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37"}
Jul 31 12:49:16.924588 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:16.924514    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="324b7836fa05d21d71e40ef557247a501455c9a0020cedf787024c8e50d67e37"
Jul 31 12:49:19.290001 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:19.289968    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"]
Jul 31 12:49:19.292461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:19.292437    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8fed071b-dxj5b"]
Jul 31 12:49:19.346159 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:19.346130    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" path="/var/lib/kubelet/pods/e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862/volumes"
Jul 31 12:49:21.952926 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:21.952889    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:49:27.565907 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:27.565870    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:49:49.008859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.008783    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"]
Jul 31 12:49:49.009260 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.009202    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" containerName="revoke-keys"
Jul 31 12:49:49.009260 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.009214    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" containerName="revoke-keys"
Jul 31 12:49:49.009340 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.009299    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ac57bd-2ffa-4829-8f9a-7b7dd20e5862" containerName="revoke-keys"
Jul 31 12:49:49.012718 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.012693    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:49.014889 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.014869    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 12:49:49.015017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.014960    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 12:49:49.019888 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.019870    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"]
Jul 31 12:49:49.067768 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.067736    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt\" (UID: \"f1ca43e4-eb90-4124-90be-e8050d33a633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:49.168859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.168830    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt\" (UID: \"f1ca43e4-eb90-4124-90be-e8050d33a633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:49.169448 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.169429    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt\" (UID: \"f1ca43e4-eb90-4124-90be-e8050d33a633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:49.325608 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.325546    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:49.454837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:49.454810    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"]
Jul 31 12:49:49.456236 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:49:49.456210    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ca43e4_eb90_4124_90be_e8050d33a633.slice/crio-f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf WatchSource:0}: Error finding container f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf: Status 404 returned error can't find the container with id f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf
Jul 31 12:49:50.111242 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:50.111105    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt" event={"ID":"f1ca43e4-eb90-4124-90be-e8050d33a633","Type":"ContainerStarted","Data":"74c55192deeab3f4c69a1febc3038adcef2ea93cd6c202a126d53ef15f8ae91d"}
Jul 31 12:49:50.111242 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:50.111161    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt" event={"ID":"f1ca43e4-eb90-4124-90be-e8050d33a633","Type":"ContainerStarted","Data":"f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf"}
Jul 31 12:49:50.125654 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:50.125612    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt" podStartSLOduration=2.125596802 podStartE2EDuration="2.125596802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:49:50.123663105 +0000 UTC m=+2077.355527856" watchObservedRunningTime="2026-07-31 12:49:50.125596802 +0000 UTC m=+2077.357461580"
Jul 31 12:49:51.117672 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:51.117633    2532 generic.go:356] "Generic (PLEG): container finished" podID="f1ca43e4-eb90-4124-90be-e8050d33a633" containerID="74c55192deeab3f4c69a1febc3038adcef2ea93cd6c202a126d53ef15f8ae91d" exitCode=0
Jul 31 12:49:51.118035 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:51.117684    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt" event={"ID":"f1ca43e4-eb90-4124-90be-e8050d33a633","Type":"ContainerDied","Data":"74c55192deeab3f4c69a1febc3038adcef2ea93cd6c202a126d53ef15f8ae91d"}
Jul 31 12:49:52.251258 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:52.251233    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:52.299292 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:52.299267    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca\") pod \"f1ca43e4-eb90-4124-90be-e8050d33a633\" (UID: \"f1ca43e4-eb90-4124-90be-e8050d33a633\") "
Jul 31 12:49:52.299625 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:52.299606    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1ca43e4-eb90-4124-90be-e8050d33a633" (UID: "f1ca43e4-eb90-4124-90be-e8050d33a633"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:49:52.400523 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:52.400466    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ca43e4-eb90-4124-90be-e8050d33a633-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:49:53.129583 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:53.129553    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt" event={"ID":"f1ca43e4-eb90-4124-90be-e8050d33a633","Type":"ContainerDied","Data":"f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf"}
Jul 31 12:49:53.129583 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:53.129584    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"
Jul 31 12:49:53.129802 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:53.129602    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f19fd78db69876a8d0991f188f74971a607fb2463ced03d815b9e201484029cf"
Jul 31 12:49:55.942717 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:55.942673    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"]
Jul 31 12:49:55.944807 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:55.944784    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46d34f7d-5jhdt"]
Jul 31 12:49:57.349714 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:57.349682    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ca43e4-eb90-4124-90be-e8050d33a633" path="/var/lib/kubelet/pods/f1ca43e4-eb90-4124-90be-e8050d33a633/volumes"
Jul 31 12:49:58.352838 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:49:58.352801    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:50:09.950189 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:50:09.950133    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:51:37.857796 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:51:37.857711    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:51:44.551690 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:51:44.551653    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:51:49.842859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:51:49.842823    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:51:58.548387 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:51:58.548354    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:52:27.106395 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.106362    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"]
Jul 31 12:52:27.106976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.106942    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ca43e4-eb90-4124-90be-e8050d33a633" containerName="revoke-keys"
Jul 31 12:52:27.106976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.106962    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ca43e4-eb90-4124-90be-e8050d33a633" containerName="revoke-keys"
Jul 31 12:52:27.107095 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.107070    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ca43e4-eb90-4124-90be-e8050d33a633" containerName="revoke-keys"
Jul 31 12:52:27.110094 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.110073    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:27.112460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.112440    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 12:52:27.112558 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.112438    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 12:52:27.117385 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.117352    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"]
Jul 31 12:52:27.229254 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.229221    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-88cd166c-j4d95\" (UID: \"0db73542-d853-46cd-b516-7a658aa19a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:27.329954 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.329931    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-88cd166c-j4d95\" (UID: \"0db73542-d853-46cd-b516-7a658aa19a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:27.330523 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.330505    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-88cd166c-j4d95\" (UID: \"0db73542-d853-46cd-b516-7a658aa19a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:27.422416 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.422361    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:27.755620 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:27.755593    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"]
Jul 31 12:52:27.757687 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:52:27.757647    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db73542_d853_46cd_b516_7a658aa19a8e.slice/crio-79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1 WatchSource:0}: Error finding container 79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1: Status 404 returned error can't find the container with id 79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1
Jul 31 12:52:28.013217 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:28.013129    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95" event={"ID":"0db73542-d853-46cd-b516-7a658aa19a8e","Type":"ContainerStarted","Data":"a60a303fed070a4ede47aeaf5cb14837f8ec536c5c25e70c65de66d9c201d596"}
Jul 31 12:52:28.013330 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:28.013212    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95" event={"ID":"0db73542-d853-46cd-b516-7a658aa19a8e","Type":"ContainerStarted","Data":"79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1"}
Jul 31 12:52:28.028264 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:28.028211    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95" podStartSLOduration=1.028197472 podStartE2EDuration="1.028197472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:52:28.026402557 +0000 UTC m=+2235.258267309" watchObservedRunningTime="2026-07-31 12:52:28.028197472 +0000 UTC m=+2235.260062217"
Jul 31 12:52:29.019784 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:29.019746    2532 generic.go:356] "Generic (PLEG): container finished" podID="0db73542-d853-46cd-b516-7a658aa19a8e" containerID="a60a303fed070a4ede47aeaf5cb14837f8ec536c5c25e70c65de66d9c201d596" exitCode=0
Jul 31 12:52:29.020219 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:29.019826    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95" event={"ID":"0db73542-d853-46cd-b516-7a658aa19a8e","Type":"ContainerDied","Data":"a60a303fed070a4ede47aeaf5cb14837f8ec536c5c25e70c65de66d9c201d596"}
Jul 31 12:52:30.155687 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:30.155658    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:30.253082 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:30.253056    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca\") pod \"0db73542-d853-46cd-b516-7a658aa19a8e\" (UID: \"0db73542-d853-46cd-b516-7a658aa19a8e\") "
Jul 31 12:52:30.253388 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:30.253367    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0db73542-d853-46cd-b516-7a658aa19a8e" (UID: "0db73542-d853-46cd-b516-7a658aa19a8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 12:52:30.354091 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:30.354033    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db73542-d853-46cd-b516-7a658aa19a8e-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:52:31.031583 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:31.031556    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"
Jul 31 12:52:31.031789 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:31.031602    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95" event={"ID":"0db73542-d853-46cd-b516-7a658aa19a8e","Type":"ContainerDied","Data":"79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1"}
Jul 31 12:52:31.031789 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:31.031651    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79adca7537d88ce68af2eeac96612a27c3a0547615647286f6f3b8a1ebb087e1"
Jul 31 12:52:37.647209 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:37.647108    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"]
Jul 31 12:52:37.649296 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:37.649275    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-88cd166c-j4d95"]
Jul 31 12:52:39.346915 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:39.346882    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0db73542-d853-46cd-b516-7a658aa19a8e" path="/var/lib/kubelet/pods/0db73542-d853-46cd-b516-7a658aa19a8e/volumes"
Jul 31 12:52:39.347297 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:52:39.347147    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:53:15.552939 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:15.552904    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:53:24.811451 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:24.811405    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:53:24.811901 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:24.811669    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" podUID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" containerName="manager" containerID="cri-o://1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f" gracePeriod=10
Jul 31 12:53:25.057250 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.057225    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:53:25.114944 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.114879    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume\") pod \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") "
Jul 31 12:53:25.115077 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.114945    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7wvv\" (UniqueName: \"kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv\") pod \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\" (UID: \"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9\") "
Jul 31 12:53:25.115269 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.115246    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" (UID: "d39f0541-ce76-4e59-ba43-ed92bfd8ccd9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:53:25.116880 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.116854    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv" (OuterVolumeSpecName: "kube-api-access-g7wvv") pod "d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" (UID: "d39f0541-ce76-4e59-ba43-ed92bfd8ccd9"). InnerVolumeSpecName "kube-api-access-g7wvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:53:25.216161 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.216135    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7wvv\" (UniqueName: \"kubernetes.io/projected/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-kube-api-access-g7wvv\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:53:25.216161 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.216159    2532 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9-extensions-socket-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:53:25.341758 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.341732    2532 generic.go:356] "Generic (PLEG): container finished" podID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" containerID="1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f" exitCode=0
Jul 31 12:53:25.341871 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.341842    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"
Jul 31 12:53:25.346920 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.346900    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" event={"ID":"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9","Type":"ContainerDied","Data":"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"}
Jul 31 12:53:25.347016 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.346936    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw" event={"ID":"d39f0541-ce76-4e59-ba43-ed92bfd8ccd9","Type":"ContainerDied","Data":"06d572bafdde79b2a814fc019d230dd3d4c2dde6291870695f7c6047574da2e0"}
Jul 31 12:53:25.347016 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.346965    2532 scope.go:117] "RemoveContainer" containerID="1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"
Jul 31 12:53:25.357502 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.357485    2532 scope.go:117] "RemoveContainer" containerID="1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"
Jul 31 12:53:25.357778 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:53:25.357760    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f\": container with ID starting with 1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f not found: ID does not exist" containerID="1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"
Jul 31 12:53:25.357834 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.357787    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f"} err="failed to get container status \"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f\": rpc error: code = NotFound desc = could not find container \"1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f\": container with ID starting with 1658f33b10b253d361aa159aa9b2824f3bafe9bb6b95352feb13063916d0511f not found: ID does not exist"
Jul 31 12:53:25.365970 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.365919    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:53:25.369383 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:25.369363    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-84qdw"]
Jul 31 12:53:27.346721 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:53:27.346689    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" path="/var/lib/kubelet/pods/d39f0541-ce76-4e59-ba43-ed92bfd8ccd9/volumes"
Jul 31 12:54:13.704600 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:13.704530    2532 scope.go:117] "RemoveContainer" containerID="5274245402c9328acb3f0fef1c161ee7776c551f5ef9a40e957611fbaef00f20"
Jul 31 12:54:40.444998 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.444967    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"]
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445360    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" containerName="manager"
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445374    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" containerName="manager"
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445389    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db73542-d853-46cd-b516-7a658aa19a8e" containerName="revoke-keys"
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445395    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db73542-d853-46cd-b516-7a658aa19a8e" containerName="revoke-keys"
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445467    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db73542-d853-46cd-b516-7a658aa19a8e" containerName="revoke-keys"
Jul 31 12:54:40.445477 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.445478    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39f0541-ce76-4e59-ba43-ed92bfd8ccd9" containerName="manager"
Jul 31 12:54:40.448577 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.448558    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.451627 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.451610    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-54ldl\""
Jul 31 12:54:40.463121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.463098    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"]
Jul 31 12:54:40.537370 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.537341    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96b8acde-96f6-430a-8f17-ac8130890767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.537520 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.537425    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfmq2\" (UniqueName: \"kubernetes.io/projected/96b8acde-96f6-430a-8f17-ac8130890767-kube-api-access-dfmq2\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.638057 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.638024    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96b8acde-96f6-430a-8f17-ac8130890767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.638216 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.638116    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfmq2\" (UniqueName: \"kubernetes.io/projected/96b8acde-96f6-430a-8f17-ac8130890767-kube-api-access-dfmq2\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.638398 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.638377    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96b8acde-96f6-430a-8f17-ac8130890767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.654180 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.654149    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfmq2\" (UniqueName: \"kubernetes.io/projected/96b8acde-96f6-430a-8f17-ac8130890767-kube-api-access-dfmq2\") pod \"kuadrant-operator-controller-manager-68cf589589-ptxn7\" (UID: \"96b8acde-96f6-430a-8f17-ac8130890767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.758476 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.758445    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:40.910684 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:40.910656    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"]
Jul 31 12:54:40.912005 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:54:40.911979    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b8acde_96f6_430a_8f17_ac8130890767.slice/crio-53c8ef435e573635503f43cf306d85171faddb8ba5fb17ffd6552cb61a21a55e WatchSource:0}: Error finding container 53c8ef435e573635503f43cf306d85171faddb8ba5fb17ffd6552cb61a21a55e: Status 404 returned error can't find the container with id 53c8ef435e573635503f43cf306d85171faddb8ba5fb17ffd6552cb61a21a55e
Jul 31 12:54:41.764953 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:41.764918    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7" event={"ID":"96b8acde-96f6-430a-8f17-ac8130890767","Type":"ContainerStarted","Data":"d2b7fd5fd3627ab5bdb0265748b42dc0f757a726fec6a9c840bd51291178bafb"}
Jul 31 12:54:41.765348 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:41.764975    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7" event={"ID":"96b8acde-96f6-430a-8f17-ac8130890767","Type":"ContainerStarted","Data":"53c8ef435e573635503f43cf306d85171faddb8ba5fb17ffd6552cb61a21a55e"}
Jul 31 12:54:41.765348 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:41.765073    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:54:41.787832 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:41.787782    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7" podStartSLOduration=1.7877685589999999 podStartE2EDuration="1.787768559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:54:41.785350421 +0000 UTC m=+2369.017215172" watchObservedRunningTime="2026-07-31 12:54:41.787768559 +0000 UTC m=+2369.019633309"
Jul 31 12:54:52.772945 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:54:52.772914    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ptxn7"
Jul 31 12:55:03.153108 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:03.153073    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:55:10.952835 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:10.952799    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:55:13.728593 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:13.728560    2532 scope.go:117] "RemoveContainer" containerID="5d9a17ae368a324cfaa70776e1aea14091fd7a0c5ec9e5c53f616ae03d9cb659"
Jul 31 12:55:29.943243 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.943210    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:55:29.946861 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.946835    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:29.950165 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.950129    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bcf3ab8b\"/\"kube-root-ca.crt\""
Jul 31 12:55:29.950165 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.950157    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bcf3ab8b\"/\"openshift-service-ca.crt\""
Jul 31 12:55:29.950366 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.950230    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bcf3ab8b\"/\"default-dockercfg-9jbdx\""
Jul 31 12:55:29.950366 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.950132    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bcf3ab8b\"/\"e2e-lifecycle-model-bcf3ab8b-kserve-self-signed-certs\""
Jul 31 12:55:29.954485 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:29.954462    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:55:30.073586 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073552    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r22ch\" (UniqueName: \"kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.073739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073599    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.073739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073646    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.073739 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073731    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.073913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073761    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.073913 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.073808    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174648 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174616    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174648 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174651    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174844 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174673    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174844 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174733    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r22ch\" (UniqueName: \"kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174844 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174772    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.174844 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.174799    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.175099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.175077    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.175156 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.175121    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.175223 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.175163    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.176950 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.176930    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.177136 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.177120    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.182419 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.182400    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r22ch\" (UniqueName: \"kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch\") pod \"e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") " pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.258567 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.258541    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:30.387957 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:30.387930    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:55:30.389233 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:55:30.389207    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod134cfc08_2393_4061_a24b_2e5b3feaa6f9.slice/crio-a60b798b982e816e645cc68781d56160ece7f7e257097b41d03e611d86899947 WatchSource:0}: Error finding container a60b798b982e816e645cc68781d56160ece7f7e257097b41d03e611d86899947: Status 404 returned error can't find the container with id a60b798b982e816e645cc68781d56160ece7f7e257097b41d03e611d86899947
Jul 31 12:55:31.044614 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:31.044573    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" event={"ID":"134cfc08-2393-4061-a24b-2e5b3feaa6f9","Type":"ContainerStarted","Data":"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"}
Jul 31 12:55:31.044976 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:31.044627    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" event={"ID":"134cfc08-2393-4061-a24b-2e5b3feaa6f9","Type":"ContainerStarted","Data":"a60b798b982e816e645cc68781d56160ece7f7e257097b41d03e611d86899947"}
Jul 31 12:55:31.063679 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:31.063637    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" podStartSLOduration=2.063623336 podStartE2EDuration="2.063623336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:55:31.061400623 +0000 UTC m=+2418.293265366" watchObservedRunningTime="2026-07-31 12:55:31.063623336 +0000 UTC m=+2418.295488126"
Jul 31 12:55:32.046216 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:32.046181    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:55:40.258991 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:40.258910    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:40.258991 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:40.258964    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:40.271254 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:40.271228    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:41.113649 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:41.113621    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:55:48.343409 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:55:48.343377    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:56:10.053420 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:10.053379    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:56:13.749848 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:13.749802    2532 scope.go:117] "RemoveContainer" containerID="74c55192deeab3f4c69a1febc3038adcef2ea93cd6c202a126d53ef15f8ae91d"
Jul 31 12:56:13.758746 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:13.758726    2532 scope.go:117] "RemoveContainer" containerID="919bea6bd286899651209d07e219bb321a1d81dafc8a73f8efe6d64f19c7de81"
Jul 31 12:56:15.452884 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:15.452851    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:56:20.757911 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:20.757870    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:56:35.777428 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:35.777388    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" podUID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" containerName="main" containerID="cri-o://f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a" gracePeriod=45
Jul 31 12:56:36.023648 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.023623    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:56:36.075573 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075504    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.075704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075579    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r22ch\" (UniqueName: \"kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.075704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075600    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.075704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075632    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.075704 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075675    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.075917 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075730    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache\") pod \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\" (UID: \"134cfc08-2393-4061-a24b-2e5b3feaa6f9\") "
Jul 31 12:56:36.076141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075976    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:56:36.076141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.075988    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home" (OuterVolumeSpecName: "home") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:56:36.076141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.076108    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:56:36.076141 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.076116    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.077736 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.077712    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm" (OuterVolumeSpecName: "dshm") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 12:56:36.077736 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.077725    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch" (OuterVolumeSpecName: "kube-api-access-r22ch") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "kube-api-access-r22ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 12:56:36.077862 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.077741    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "134cfc08-2393-4061-a24b-2e5b3feaa6f9" (UID: "134cfc08-2393-4061-a24b-2e5b3feaa6f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 12:56:36.177461 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.177428    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.177615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.177478    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r22ch\" (UniqueName: \"kubernetes.io/projected/134cfc08-2393-4061-a24b-2e5b3feaa6f9-kube-api-access-r22ch\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.177615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.177494    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134cfc08-2393-4061-a24b-2e5b3feaa6f9-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.177615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.177504    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.177615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.177514    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/134cfc08-2393-4061-a24b-2e5b3feaa6f9-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 12:56:36.420837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.420746    2532 generic.go:356] "Generic (PLEG): container finished" podID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" containerID="f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a" exitCode=0
Jul 31 12:56:36.420837 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.420811    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"
Jul 31 12:56:36.421010 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.420830    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" event={"ID":"134cfc08-2393-4061-a24b-2e5b3feaa6f9","Type":"ContainerDied","Data":"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"}
Jul 31 12:56:36.421010 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.420876    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9" event={"ID":"134cfc08-2393-4061-a24b-2e5b3feaa6f9","Type":"ContainerDied","Data":"a60b798b982e816e645cc68781d56160ece7f7e257097b41d03e611d86899947"}
Jul 31 12:56:36.421010 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.420890    2532 scope.go:117] "RemoveContainer" containerID="f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"
Jul 31 12:56:36.430353 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.430335    2532 scope.go:117] "RemoveContainer" containerID="f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"
Jul 31 12:56:36.430600 ip-10-0-141-165 kubenswrapper[2532]: E0731 12:56:36.430571    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a\": container with ID starting with f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a not found: ID does not exist" containerID="f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"
Jul 31 12:56:36.430663 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.430613    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a"} err="failed to get container status \"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a\": rpc error: code = NotFound desc = could not find container \"f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a\": container with ID starting with f2b6a18a6449b3b4c51a13d43517cfceb9aebdac9495cca84e52174530b4d62a not found: ID does not exist"
Jul 31 12:56:36.442285 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.442264    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:56:36.445780 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:36.445758    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-bcf3ab8b/e2e-lifecycle-model-bcf3ab8b-kserve-67db4cd975-pdzx9"]
Jul 31 12:56:37.346365 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:56:37.346331    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" path="/var/lib/kubelet/pods/134cfc08-2393-4061-a24b-2e5b3feaa6f9/volumes"
Jul 31 12:57:05.463286 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:57:05.463208    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:57:18.173528 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:57:18.173491    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:57:24.201113 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:57:24.201075    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:57:31.863345 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:57:31.863305    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:58:15.455111 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:15.455076    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:58:41.805529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.805449    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 12:58:41.806055 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.806039    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" containerName="main"
Jul 31 12:58:41.806099 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.806058    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" containerName="main"
Jul 31 12:58:41.806192 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.806159    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="134cfc08-2393-4061-a24b-2e5b3feaa6f9" containerName="main"
Jul 31 12:58:41.809615 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.809597    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.812230 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.812204    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 12:58:41.812334 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.812244    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e614c6\""
Jul 31 12:58:41.813142 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.813124    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e614c6-dockercfg-djks9\""
Jul 31 12:58:41.813267 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.813152    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 12:58:41.817489 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.817363    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 12:58:41.886894 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.886864    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x8rk\" (UniqueName: \"kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.887059 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.886924    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.988076 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.988029    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x8rk\" (UniqueName: \"kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.988289 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.988130    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.988812 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.988791    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:41.996617 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:41.996582    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x8rk\" (UniqueName: \"kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk\") pod \"payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:42.122043 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:42.121959    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:42.262281 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:42.262251    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 12:58:42.263083 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:58:42.263042    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4334fb45_233d_4bb6_9c06_6c01b08d6614.slice/crio-cbef631f915350470e3e64bf50a90628e2f8e864c603d3bbeea947e71bc1240a WatchSource:0}: Error finding container cbef631f915350470e3e64bf50a90628e2f8e864c603d3bbeea947e71bc1240a: Status 404 returned error can't find the container with id cbef631f915350470e3e64bf50a90628e2f8e864c603d3bbeea947e71bc1240a
Jul 31 12:58:42.265000 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:42.264985    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 12:58:43.152637 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:43.152591    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" event={"ID":"4334fb45-233d-4bb6-9c06-6c01b08d6614","Type":"ContainerStarted","Data":"cbef631f915350470e3e64bf50a90628e2f8e864c603d3bbeea947e71bc1240a"}
Jul 31 12:58:43.957986 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:43.957927    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:58:44.159662 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:44.159569    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" event={"ID":"4334fb45-233d-4bb6-9c06-6c01b08d6614","Type":"ContainerStarted","Data":"de56c0a38b0fcb3ab3d8265286b27d219dd09912ef1da64bc0511712ce3d511a"}
Jul 31 12:58:44.159662 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:44.159652    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:58:44.181289 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:44.181242    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" podStartSLOduration=1.594311762 podStartE2EDuration="3.181227115s" totalImagesPullingTime="1.586915353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:58:44.178693036 +0000 UTC m=+2611.410557788" watchObservedRunningTime="2026-07-31 12:58:44.181227115 +0000 UTC m=+2611.413091866"
Jul 31 12:58:55.167351 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:58:55.167316    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 12:59:08.748944 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.748900    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 12:59:08.752527 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.752506    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.754859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.754840    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20ab27\"/\"kube-root-ca.crt\""
Jul 31 12:59:08.755596 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.755581    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20ab27\"/\"openshift-service-ca.crt\""
Jul 31 12:59:08.755670 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.755630    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20ab27\"/\"test-model-20ab27-kserve-self-signed-certs\""
Jul 31 12:59:08.755785 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.755767    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20ab27\"/\"default-dockercfg-m7f6s\""
Jul 31 12:59:08.761571 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.761547    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 12:59:08.854336 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854307    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.854494 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854405    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.854494 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854428    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.854574 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854487    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.854574 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854527    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.854574 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.854550    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p689m\" (UniqueName: \"kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955275 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955232    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955328    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955348    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955384    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955415    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955460 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955441    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p689m\" (UniqueName: \"kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955795 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955768    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955869 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955781    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.955869 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.955815    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.957567 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.957541    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.957711 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.957696    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:08.964150 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:08.964128    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p689m\" (UniqueName: \"kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m\") pod \"test-model-20ab27-kserve-64d79f7b5c-crl6b\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") " pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:09.063338 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:09.063253    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:09.193977 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:09.193930    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 12:59:09.195995 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:59:09.195961    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae482784_846e_41f6_a84d_61dd65c49d5c.slice/crio-08885e962845172be6b0bdb8039d40fb8232ab021b63aa74f4313c59cffbaa27 WatchSource:0}: Error finding container 08885e962845172be6b0bdb8039d40fb8232ab021b63aa74f4313c59cffbaa27: Status 404 returned error can't find the container with id 08885e962845172be6b0bdb8039d40fb8232ab021b63aa74f4313c59cffbaa27
Jul 31 12:59:09.305985 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:09.305945    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" event={"ID":"ae482784-846e-41f6-a84d-61dd65c49d5c","Type":"ContainerStarted","Data":"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"}
Jul 31 12:59:09.306207 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:09.305995    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" event={"ID":"ae482784-846e-41f6-a84d-61dd65c49d5c","Type":"ContainerStarted","Data":"08885e962845172be6b0bdb8039d40fb8232ab021b63aa74f4313c59cffbaa27"}
Jul 31 12:59:09.323541 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:09.323448    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" podStartSLOduration=1.323430297 podStartE2EDuration="1.323430297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:59:09.322061345 +0000 UTC m=+2636.553926097" watchObservedRunningTime="2026-07-31 12:59:09.323430297 +0000 UTC m=+2636.555295051"
Jul 31 12:59:11.257017 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:11.256983    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:59:13.805001 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:13.804962    2532 scope.go:117] "RemoveContainer" containerID="a60a303fed070a4ede47aeaf5cb14837f8ec536c5c25e70c65de66d9c201d596"
Jul 31 12:59:18.246745 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:18.246713    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:59:19.063529 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:19.063495    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:19.063725 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:19.063544    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:19.075537 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:19.075504    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:19.373261 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:19.373178    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 12:59:27.557231 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.557199    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 12:59:27.561261 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.561238    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.564420 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.564396    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e614c6\"/\"test-model-e614c6-kserve-self-signed-certs\""
Jul 31 12:59:27.564549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.564440    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e614c6\"/\"kube-root-ca.crt\""
Jul 31 12:59:27.564549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.564441    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e614c6\"/\"default-dockercfg-vbbxt\""
Jul 31 12:59:27.564549 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.564402    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e614c6\"/\"openshift-service-ca.crt\""
Jul 31 12:59:27.571601 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.571576    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 12:59:27.639506 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639474    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.639678 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639524    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.639678 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639578    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.639678 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639624    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.639678 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639658    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc6qx\" (UniqueName: \"kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.639859 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.639690    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740411 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740373    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740420    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc6qx\" (UniqueName: \"kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740453    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740498    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740539    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740582 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740572    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740840 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740813    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740897 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740850    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.740897 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.740883    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.742727 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.742698    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.743065 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.743047    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.748583 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.748562    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc6qx\" (UniqueName: \"kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx\") pod \"test-model-e614c6-kserve-66c6c675d7-9l7dt\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") " pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:27.871121 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:27.871046    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:28.001966 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:28.001935    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 12:59:28.003862 ip-10-0-141-165 kubenswrapper[2532]: W0731 12:59:28.003835    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod623fc1bd_0c59_4511_b875_23238c562edf.slice/crio-fd6ed36e7cba1cfc92646b2cffc095b2f914839bcb01630c25c3222bc4657567 WatchSource:0}: Error finding container fd6ed36e7cba1cfc92646b2cffc095b2f914839bcb01630c25c3222bc4657567: Status 404 returned error can't find the container with id fd6ed36e7cba1cfc92646b2cffc095b2f914839bcb01630c25c3222bc4657567
Jul 31 12:59:28.411905 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:28.411861    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" event={"ID":"623fc1bd-0c59-4511-b875-23238c562edf","Type":"ContainerStarted","Data":"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"}
Jul 31 12:59:28.412071 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:28.411916    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" event={"ID":"623fc1bd-0c59-4511-b875-23238c562edf","Type":"ContainerStarted","Data":"fd6ed36e7cba1cfc92646b2cffc095b2f914839bcb01630c25c3222bc4657567"}
Jul 31 12:59:28.428855 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:28.428801    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" podStartSLOduration=1.4287867109999999 podStartE2EDuration="1.428786711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 12:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 12:59:28.42659439 +0000 UTC m=+2655.658459140" watchObservedRunningTime="2026-07-31 12:59:28.428786711 +0000 UTC m=+2655.660651508"
Jul 31 12:59:30.452456 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:30.452418    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 12:59:37.871938 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:37.871895    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:37.871938 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:37.871944    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:37.884224 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:37.884195    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:38.480517 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:38.480483    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 12:59:40.151830 ip-10-0-141-165 kubenswrapper[2532]: I0731 12:59:40.151791    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:00:09.209927 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:09.209848    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 13:00:10.125674 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:10.125631    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 13:00:12.578994 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:12.578957    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:00:22.762447 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:22.762415    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:00:24.228485 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.228438    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" podUID="ae482784-846e-41f6-a84d-61dd65c49d5c" containerName="main" containerID="cri-o://89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff" gracePeriod=45
Jul 31 13:00:24.483513 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.483456    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 13:00:24.590003 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.589974    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590003 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590003    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590212 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590032    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p689m\" (UniqueName: \"kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590212 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590063    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590212 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590149    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590366 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590211    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir\") pod \"ae482784-846e-41f6-a84d-61dd65c49d5c\" (UID: \"ae482784-846e-41f6-a84d-61dd65c49d5c\") "
Jul 31 13:00:24.590366 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590297    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:24.590505 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590463    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home" (OuterVolumeSpecName: "home") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:24.590566 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590518    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:24.590621 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590604    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.590666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590627    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.590666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.590643    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.592259 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.592235    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m" (OuterVolumeSpecName: "kube-api-access-p689m") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "kube-api-access-p689m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:00:24.592259 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.592236    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm" (OuterVolumeSpecName: "dshm") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:24.592433 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.592267    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae482784-846e-41f6-a84d-61dd65c49d5c" (UID: "ae482784-846e-41f6-a84d-61dd65c49d5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:00:24.698898 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.698869    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae482784-846e-41f6-a84d-61dd65c49d5c-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.699097 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.699076    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p689m\" (UniqueName: \"kubernetes.io/projected/ae482784-846e-41f6-a84d-61dd65c49d5c-kube-api-access-p689m\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.699152 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.699105    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae482784-846e-41f6-a84d-61dd65c49d5c-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:24.741787 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.741721    2532 generic.go:356] "Generic (PLEG): container finished" podID="ae482784-846e-41f6-a84d-61dd65c49d5c" containerID="89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff" exitCode=0
Jul 31 13:00:24.741923 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.741783    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"
Jul 31 13:00:24.741923 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.741816    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" event={"ID":"ae482784-846e-41f6-a84d-61dd65c49d5c","Type":"ContainerDied","Data":"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"}
Jul 31 13:00:24.741923 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.741859    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b" event={"ID":"ae482784-846e-41f6-a84d-61dd65c49d5c","Type":"ContainerDied","Data":"08885e962845172be6b0bdb8039d40fb8232ab021b63aa74f4313c59cffbaa27"}
Jul 31 13:00:24.741923 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.741877    2532 scope.go:117] "RemoveContainer" containerID="89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"
Jul 31 13:00:24.751165 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.751144    2532 scope.go:117] "RemoveContainer" containerID="89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"
Jul 31 13:00:24.751486 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:00:24.751467    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff\": container with ID starting with 89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff not found: ID does not exist" containerID="89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"
Jul 31 13:00:24.751563 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.751493    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff"} err="failed to get container status \"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff\": rpc error: code = NotFound desc = could not find container \"89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff\": container with ID starting with 89d45fccacd2202e3e661d14325ad5b501d9506b9c03a507ef85f301f35582ff not found: ID does not exist"
Jul 31 13:00:24.764277 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.764252    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 13:00:24.767642 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:24.767622    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-20ab27/test-model-20ab27-kserve-64d79f7b5c-crl6b"]
Jul 31 13:00:25.145276 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.145189    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" podUID="623fc1bd-0c59-4511-b875-23238c562edf" containerName="main" containerID="cri-o://09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff" gracePeriod=45
Jul 31 13:00:25.349008 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.348978    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae482784-846e-41f6-a84d-61dd65c49d5c" path="/var/lib/kubelet/pods/ae482784-846e-41f6-a84d-61dd65c49d5c/volumes"
Jul 31 13:00:25.399166 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.399112    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 13:00:25.405102 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405080    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405222 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405123    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405222 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405141    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405222 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405201    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc6qx\" (UniqueName: \"kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405377 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405286    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405377 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405321    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache\") pod \"623fc1bd-0c59-4511-b875-23238c562edf\" (UID: \"623fc1bd-0c59-4511-b875-23238c562edf\") "
Jul 31 13:00:25.405483 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405371    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:25.405483 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405469    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home" (OuterVolumeSpecName: "home") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:25.405645 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405618    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache" (OuterVolumeSpecName: "model-cache") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:25.405751 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405717    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.405751 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405734    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.405751 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.405749    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.407333 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.407308    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:00:25.407418 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.407328    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx" (OuterVolumeSpecName: "kube-api-access-cc6qx") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "kube-api-access-cc6qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:00:25.407582 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.407560    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm" (OuterVolumeSpecName: "dshm") pod "623fc1bd-0c59-4511-b875-23238c562edf" (UID: "623fc1bd-0c59-4511-b875-23238c562edf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:00:25.506523 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.506491    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623fc1bd-0c59-4511-b875-23238c562edf-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.506523 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.506525    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc6qx\" (UniqueName: \"kubernetes.io/projected/623fc1bd-0c59-4511-b875-23238c562edf-kube-api-access-cc6qx\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.506679 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.506540    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623fc1bd-0c59-4511-b875-23238c562edf-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:25.748484 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.748450    2532 generic.go:356] "Generic (PLEG): container finished" podID="623fc1bd-0c59-4511-b875-23238c562edf" containerID="09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff" exitCode=0
Jul 31 13:00:25.748667 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.748503    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"
Jul 31 13:00:25.748667 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.748536    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" event={"ID":"623fc1bd-0c59-4511-b875-23238c562edf","Type":"ContainerDied","Data":"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"}
Jul 31 13:00:25.748667 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.748580    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt" event={"ID":"623fc1bd-0c59-4511-b875-23238c562edf","Type":"ContainerDied","Data":"fd6ed36e7cba1cfc92646b2cffc095b2f914839bcb01630c25c3222bc4657567"}
Jul 31 13:00:25.748667 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.748599    2532 scope.go:117] "RemoveContainer" containerID="09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"
Jul 31 13:00:25.759972 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.759957    2532 scope.go:117] "RemoveContainer" containerID="09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"
Jul 31 13:00:25.760385 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:00:25.760358    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff\": container with ID starting with 09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff not found: ID does not exist" containerID="09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"
Jul 31 13:00:25.760460 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.760405    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff"} err="failed to get container status \"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff\": rpc error: code = NotFound desc = could not find container \"09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff\": container with ID starting with 09eeed1b143a9759d91f8765e94fb093327eed24096a766c7f4f1747abf8f3ff not found: ID does not exist"
Jul 31 13:00:25.772050 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.772023    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 13:00:25.778981 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:25.778957    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e614c6/test-model-e614c6-kserve-66c6c675d7-9l7dt"]
Jul 31 13:00:27.346099 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:27.346059    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="623fc1bd-0c59-4511-b875-23238c562edf" path="/var/lib/kubelet/pods/623fc1bd-0c59-4511-b875-23238c562edf/volumes"
Jul 31 13:00:38.670325 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:38.670286    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 13:00:38.670760 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:38.670531    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" podUID="4334fb45-233d-4bb6-9c06-6c01b08d6614" containerName="payload-processing" containerID="cri-o://de56c0a38b0fcb3ab3d8265286b27d219dd09912ef1da64bc0511712ce3d511a" gracePeriod=30
Jul 31 13:00:38.824088 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:38.824051    2532 generic.go:356] "Generic (PLEG): container finished" podID="4334fb45-233d-4bb6-9c06-6c01b08d6614" containerID="de56c0a38b0fcb3ab3d8265286b27d219dd09912ef1da64bc0511712ce3d511a" exitCode=0
Jul 31 13:00:38.824320 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:38.824288    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" event={"ID":"4334fb45-233d-4bb6-9c06-6c01b08d6614","Type":"ContainerDied","Data":"de56c0a38b0fcb3ab3d8265286b27d219dd09912ef1da64bc0511712ce3d511a"}
Jul 31 13:00:38.927523 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:38.927460    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 13:00:39.021938 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.021907    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume\") pod \"4334fb45-233d-4bb6-9c06-6c01b08d6614\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") "
Jul 31 13:00:39.022082 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.021948    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x8rk\" (UniqueName: \"kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk\") pod \"4334fb45-233d-4bb6-9c06-6c01b08d6614\" (UID: \"4334fb45-233d-4bb6-9c06-6c01b08d6614\") "
Jul 31 13:00:39.022324 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.022304    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4334fb45-233d-4bb6-9c06-6c01b08d6614" (UID: "4334fb45-233d-4bb6-9c06-6c01b08d6614"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:00:39.023899 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.023879    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk" (OuterVolumeSpecName: "kube-api-access-6x8rk") pod "4334fb45-233d-4bb6-9c06-6c01b08d6614" (UID: "4334fb45-233d-4bb6-9c06-6c01b08d6614"). InnerVolumeSpecName "kube-api-access-6x8rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:00:39.123353 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.123323    2532 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4334fb45-233d-4bb6-9c06-6c01b08d6614-plugins-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:39.123353 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.123349    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x8rk\" (UniqueName: \"kubernetes.io/projected/4334fb45-233d-4bb6-9c06-6c01b08d6614-kube-api-access-6x8rk\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:00:39.831437 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.831406    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"
Jul 31 13:00:39.831839 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.831405    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq" event={"ID":"4334fb45-233d-4bb6-9c06-6c01b08d6614","Type":"ContainerDied","Data":"cbef631f915350470e3e64bf50a90628e2f8e864c603d3bbeea947e71bc1240a"}
Jul 31 13:00:39.831839 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.831542    2532 scope.go:117] "RemoveContainer" containerID="de56c0a38b0fcb3ab3d8265286b27d219dd09912ef1da64bc0511712ce3d511a"
Jul 31 13:00:39.849715 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.849693    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 13:00:39.853790 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:39.853769    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e614c6-6798f4dff4-57mmq"]
Jul 31 13:00:40.558506 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:40.558471    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:00:41.345788 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:41.345759    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4334fb45-233d-4bb6-9c06-6c01b08d6614" path="/var/lib/kubelet/pods/4334fb45-233d-4bb6-9c06-6c01b08d6614/volumes"
Jul 31 13:00:53.553985 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:00:53.553949    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:01:20.767059 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767014    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:01:20.767540 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767522    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4334fb45-233d-4bb6-9c06-6c01b08d6614" containerName="payload-processing"
Jul 31 13:01:20.767593 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767544    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="4334fb45-233d-4bb6-9c06-6c01b08d6614" containerName="payload-processing"
Jul 31 13:01:20.767593 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767570    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623fc1bd-0c59-4511-b875-23238c562edf" containerName="main"
Jul 31 13:01:20.767593 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767579    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="623fc1bd-0c59-4511-b875-23238c562edf" containerName="main"
Jul 31 13:01:20.767687 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767602    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae482784-846e-41f6-a84d-61dd65c49d5c" containerName="main"
Jul 31 13:01:20.767687 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767610    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae482784-846e-41f6-a84d-61dd65c49d5c" containerName="main"
Jul 31 13:01:20.767687 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767676    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="623fc1bd-0c59-4511-b875-23238c562edf" containerName="main"
Jul 31 13:01:20.767687 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767683    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae482784-846e-41f6-a84d-61dd65c49d5c" containerName="main"
Jul 31 13:01:20.767812 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.767690    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="4334fb45-233d-4bb6-9c06-6c01b08d6614" containerName="payload-processing"
Jul 31 13:01:20.771984 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.771965    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:20.776835 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.776807    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 13:01:20.777094 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.777079    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 13:01:20.777162 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.777099    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b21077-dockercfg-gbsn4\""
Jul 31 13:01:20.777232 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.777099    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b21077\""
Jul 31 13:01:20.785716 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.785682    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:01:20.891478 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.891436    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:20.891673 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.891517    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hsf2\" (UniqueName: \"kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:20.992401 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.992354    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:20.992590 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.992458    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hsf2\" (UniqueName: \"kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:20.993320 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:20.993294    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:21.001069 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:21.001038    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hsf2\" (UniqueName: \"kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2\") pod \"payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:21.082996 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:21.082910    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:21.433664 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:21.433634    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:01:21.435119 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:01:21.435092    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a14f3a1_8a9d_4563_bb43_24a03e6f28c6.slice/crio-62ce138fb7d823b35c1b150ea2754596bf806e99cbd20b98f8abfea23152ab31 WatchSource:0}: Error finding container 62ce138fb7d823b35c1b150ea2754596bf806e99cbd20b98f8abfea23152ab31: Status 404 returned error can't find the container with id 62ce138fb7d823b35c1b150ea2754596bf806e99cbd20b98f8abfea23152ab31
Jul 31 13:01:22.070274 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:22.070234    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" event={"ID":"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6","Type":"ContainerStarted","Data":"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"}
Jul 31 13:01:22.070666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:22.070301    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" event={"ID":"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6","Type":"ContainerStarted","Data":"62ce138fb7d823b35c1b150ea2754596bf806e99cbd20b98f8abfea23152ab31"}
Jul 31 13:01:22.070666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:22.070412    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:01:22.088246 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:22.088199    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" podStartSLOduration=2.088165329 podStartE2EDuration="2.088165329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:01:22.08646957 +0000 UTC m=+2769.318334324" watchObservedRunningTime="2026-07-31 13:01:22.088165329 +0000 UTC m=+2769.320030081"
Jul 31 13:01:22.351993 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:22.351890    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:01:33.078795 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:01:33.078758    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:02:04.748672 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:04.748624    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:02:24.700651 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:24.700619    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:02:24.701117 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:24.700831    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" podUID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" containerName="payload-processing" containerID="cri-o://e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684" gracePeriod=30
Jul 31 13:02:24.958505 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:24.958484    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:02:25.100728 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.100692    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume\") pod \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") "
Jul 31 13:02:25.100902 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.100738    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hsf2\" (UniqueName: \"kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2\") pod \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\" (UID: \"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6\") "
Jul 31 13:02:25.101126 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.101099    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" (UID: "3a14f3a1-8a9d-4563-bb43-24a03e6f28c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:02:25.102872 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.102843    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2" (OuterVolumeSpecName: "kube-api-access-4hsf2") pod "3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" (UID: "3a14f3a1-8a9d-4563-bb43-24a03e6f28c6"). InnerVolumeSpecName "kube-api-access-4hsf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:02:25.201778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.201753    2532 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-plugins-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:02:25.201778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.201776    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hsf2\" (UniqueName: \"kubernetes.io/projected/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6-kube-api-access-4hsf2\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:02:25.435370 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.435329    2532 generic.go:356] "Generic (PLEG): container finished" podID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" containerID="e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684" exitCode=0
Jul 31 13:02:25.435544 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.435378    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"
Jul 31 13:02:25.435544 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.435419    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" event={"ID":"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6","Type":"ContainerDied","Data":"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"}
Jul 31 13:02:25.435544 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.435480    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl" event={"ID":"3a14f3a1-8a9d-4563-bb43-24a03e6f28c6","Type":"ContainerDied","Data":"62ce138fb7d823b35c1b150ea2754596bf806e99cbd20b98f8abfea23152ab31"}
Jul 31 13:02:25.435544 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.435508    2532 scope.go:117] "RemoveContainer" containerID="e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"
Jul 31 13:02:25.444818 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.444800    2532 scope.go:117] "RemoveContainer" containerID="e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"
Jul 31 13:02:25.445082 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:02:25.445057    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684\": container with ID starting with e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684 not found: ID does not exist" containerID="e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"
Jul 31 13:02:25.445200 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.445088    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684"} err="failed to get container status \"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684\": rpc error: code = NotFound desc = could not find container \"e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684\": container with ID starting with e33b951d95c038dc34640826e32287950abfcfe3a6982028d81239a8689a4684 not found: ID does not exist"
Jul 31 13:02:25.452781 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.452759    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:02:25.456532 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:25.456505    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b21077-74c7c7b54d-c9ljl"]
Jul 31 13:02:26.752050 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:26.752011    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:02:27.347121 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:27.347089    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" path="/var/lib/kubelet/pods/3a14f3a1-8a9d-4563-bb43-24a03e6f28c6/volumes"
Jul 31 13:02:38.947227 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:02:38.947186    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:03:07.526623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.526543    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:03:07.526987 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.526949    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" containerName="payload-processing"
Jul 31 13:03:07.526987 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.526961    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" containerName="payload-processing"
Jul 31 13:03:07.527065 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.527040    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a14f3a1-8a9d-4563-bb43-24a03e6f28c6" containerName="payload-processing"
Jul 31 13:03:07.530227 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.530205    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.532660 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.532638    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-e55b67-dockercfg-49l45\""
Jul 31 13:03:07.532760 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.532702    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-e55b67\""
Jul 31 13:03:07.533476 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.533459    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 13:03:07.533598 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.533505    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 13:03:07.538375 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.538134    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:03:07.693592 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.693555    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.693782 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.693602    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzjx\" (UniqueName: \"kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.795047 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.794952    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.795047 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.794997    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzjx\" (UniqueName: \"kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.795739 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.795713    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.802781 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.802747    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzjx\" (UniqueName: \"kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx\") pod \"payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.842612 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.842583    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:07.984092 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:07.984065    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:03:07.985989 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:03:07.985959    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd4a0b0_5adf_4e7d_aed2_548a6b2b65f5.slice/crio-73791f5849fa7c3058b0c44c2dde9f0af17c037032851499747b322658193e5f WatchSource:0}: Error finding container 73791f5849fa7c3058b0c44c2dde9f0af17c037032851499747b322658193e5f: Status 404 returned error can't find the container with id 73791f5849fa7c3058b0c44c2dde9f0af17c037032851499747b322658193e5f
Jul 31 13:03:08.684550 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:08.684509    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" event={"ID":"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5","Type":"ContainerStarted","Data":"d5b7e1dfcb93bc49d1cda38960bf595bb0dd02df986c02427e8c6b7bb36fd891"}
Jul 31 13:03:08.684946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:08.684573    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" event={"ID":"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5","Type":"ContainerStarted","Data":"73791f5849fa7c3058b0c44c2dde9f0af17c037032851499747b322658193e5f"}
Jul 31 13:03:08.684946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:08.684719    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:08.702818 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:08.702756    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" podStartSLOduration=1.7027404430000002 podStartE2EDuration="1.702740443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:03:08.700753605 +0000 UTC m=+2875.932618358" watchObservedRunningTime="2026-07-31 13:03:08.702740443 +0000 UTC m=+2875.934605194"
Jul 31 13:03:09.647259 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:09.647222    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:03:19.693851 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:19.693815    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:03:21.544061 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.544020    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:03:21.549248 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.549223    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.551667 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.551642    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-96b3e0\"/\"kube-root-ca.crt\""
Jul 31 13:03:21.552851 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.552827    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-96b3e0\"/\"openshift-service-ca.crt\""
Jul 31 13:03:21.552971 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.552827    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-96b3e0\"/\"auth-test-model-96b3e0-kserve-self-signed-certs\""
Jul 31 13:03:21.552971 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.552837    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-96b3e0\"/\"default-dockercfg-9sl58\""
Jul 31 13:03:21.556890 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.556866    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:03:21.722641 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722600    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.722641 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722645    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.722873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722729    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.722873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722782    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.722873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722821    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.722991 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.722880    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4625\" (UniqueName: \"kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.823958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.823858    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.823958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.823911    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.823968    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.823997    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.824029    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.824079    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4625\" (UniqueName: \"kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824445 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.824294    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824445 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.824401    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.824445 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.824430    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.826280 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.826255    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.826417 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.826402    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.831650 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.831631    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4625\" (UniqueName: \"kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625\") pod \"auth-test-model-96b3e0-kserve-b48c68f99-q4w85\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") " pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:21.861690 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:21.861664    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:22.004520 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:22.004483    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:03:22.005094 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:03:22.005060    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25ca4e54_7f86_4509_8ac9_990c6fe88945.slice/crio-fa392d98e18e2cc268dc017c4d129667584d6ec038fd282a086ccb5d6a455c95 WatchSource:0}: Error finding container fa392d98e18e2cc268dc017c4d129667584d6ec038fd282a086ccb5d6a455c95: Status 404 returned error can't find the container with id fa392d98e18e2cc268dc017c4d129667584d6ec038fd282a086ccb5d6a455c95
Jul 31 13:03:22.777634 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:22.777597    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" event={"ID":"25ca4e54-7f86-4509-8ac9-990c6fe88945","Type":"ContainerStarted","Data":"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"}
Jul 31 13:03:22.777634 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:22.777640    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" event={"ID":"25ca4e54-7f86-4509-8ac9-990c6fe88945","Type":"ContainerStarted","Data":"fa392d98e18e2cc268dc017c4d129667584d6ec038fd282a086ccb5d6a455c95"}
Jul 31 13:03:22.792936 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:22.792878    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" podStartSLOduration=1.79286062 podStartE2EDuration="1.79286062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:03:22.792330444 +0000 UTC m=+2890.024195200" watchObservedRunningTime="2026-07-31 13:03:22.79286062 +0000 UTC m=+2890.024725387"
Jul 31 13:03:24.154688 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:24.154644    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:03:31.862400 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:31.862369    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:31.862855 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:31.862420    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:31.875160 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:31.875130    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:32.847257 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:32.847228    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:03:45.353352 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.353315    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:03:45.357226 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.357201    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.359515 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.359491    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e55b67\"/\"openshift-service-ca.crt\""
Jul 31 13:03:45.359748 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.359591    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e55b67\"/\"kube-root-ca.crt\""
Jul 31 13:03:45.360436 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.360418    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e55b67\"/\"default-dockercfg-hj927\""
Jul 31 13:03:45.360542 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.360498    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e55b67\"/\"auth-test-model-e55b67-kserve-self-signed-certs\""
Jul 31 13:03:45.366949 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.366922    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:03:45.450992 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.450953    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.451179 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.451067    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.451179 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.451098    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.451179 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.451138    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xmmw\" (UniqueName: \"kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.451326 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.451309    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.451363 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.451345    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.552754 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552717    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.552938 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552767    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.552938 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552821    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.552938 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552889    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.552938 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552919    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.553164 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.552953    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xmmw\" (UniqueName: \"kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.553267 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.553224    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.553322 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.553262    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.553322 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.553303    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.555220 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.555195    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.555522 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.555495    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.564755 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.564732    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xmmw\" (UniqueName: \"kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw\") pod \"auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") " pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.669046 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.668970    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:45.805493 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.805465    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:03:45.807516 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:03:45.807488    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod602f1c6f_68f3_479b_938a_0f74cb3c897a.slice/crio-d1f39957476c523e43496a8c633d7fb6491221ffa256c3d421da2e117abd78a9 WatchSource:0}: Error finding container d1f39957476c523e43496a8c633d7fb6491221ffa256c3d421da2e117abd78a9: Status 404 returned error can't find the container with id d1f39957476c523e43496a8c633d7fb6491221ffa256c3d421da2e117abd78a9
Jul 31 13:03:45.922985 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.922893    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" event={"ID":"602f1c6f-68f3-479b-938a-0f74cb3c897a","Type":"ContainerStarted","Data":"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"}
Jul 31 13:03:45.922985 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.922948    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" event={"ID":"602f1c6f-68f3-479b-938a-0f74cb3c897a","Type":"ContainerStarted","Data":"d1f39957476c523e43496a8c633d7fb6491221ffa256c3d421da2e117abd78a9"}
Jul 31 13:03:45.943878 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:45.943831    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" podStartSLOduration=0.943815731 podStartE2EDuration="943.815731ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:03:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:03:45.94134398 +0000 UTC m=+2913.173208733" watchObservedRunningTime="2026-07-31 13:03:45.943815731 +0000 UTC m=+2913.175680482"
Jul 31 13:03:48.245672 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:48.245634    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:03:55.670071 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:55.670035    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:55.670696 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:55.670118    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:55.682793 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:55.682772    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:03:55.992439 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:03:55.992404    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:04:14.160300 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:04:14.160264    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:04:29.650350 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:04:29.650311    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:04:37.552481 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:04:37.552403    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:04:46.360487 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:04:46.360449    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:04:56.657975 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:04:56.657937    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:05:07.749896 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:05:07.749842    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:05:23.054043 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:05:23.054005    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:05:34.462272 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:05:34.462232    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:05:48.951103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:05:48.951068    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:05:59.648967 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:05:59.648923    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:06:15.646452 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:06:15.646363    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:06:31.150838 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:06:31.150802    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:06:41.749102 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:06:41.749059    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:06:57.266707 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:06:57.266673    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:07:10.754110 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:10.754065    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:07:14.688219 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:14.688183    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:07:15.753669 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:15.753636    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:07:21.357717 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:21.357673    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:07:27.148116 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:27.148078    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:07:29.707807 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:29.707743    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" podUID="25ca4e54-7f86-4509-8ac9-990c6fe88945" containerName="main" containerID="cri-o://fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76" gracePeriod=45
Jul 31 13:07:29.957736 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:29.957713    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:07:30.091455 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091420    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4625\" (UniqueName: \"kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091476    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091514    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091583    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091621    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091814 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091650    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir\") pod \"25ca4e54-7f86-4509-8ac9-990c6fe88945\" (UID: \"25ca4e54-7f86-4509-8ac9-990c6fe88945\") "
Jul 31 13:07:30.091814 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091789    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache" (OuterVolumeSpecName: "model-cache") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:30.091937 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091849    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home" (OuterVolumeSpecName: "home") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:30.091992 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091967    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.091992 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.091979    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.092095 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.092074    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:30.094069 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.094027    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm" (OuterVolumeSpecName: "dshm") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:30.094069 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.094054    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:07:30.094300 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.094109    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625" (OuterVolumeSpecName: "kube-api-access-c4625") pod "25ca4e54-7f86-4509-8ac9-990c6fe88945" (UID: "25ca4e54-7f86-4509-8ac9-990c6fe88945"). InnerVolumeSpecName "kube-api-access-c4625". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:07:30.193381 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.193345    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.193517 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.193385    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4625\" (UniqueName: \"kubernetes.io/projected/25ca4e54-7f86-4509-8ac9-990c6fe88945-kube-api-access-c4625\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.193517 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.193399    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ca4e54-7f86-4509-8ac9-990c6fe88945-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.193517 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.193412    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ca4e54-7f86-4509-8ac9-990c6fe88945-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:30.242531 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.242498    2532 generic.go:356] "Generic (PLEG): container finished" podID="25ca4e54-7f86-4509-8ac9-990c6fe88945" containerID="fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76" exitCode=0
Jul 31 13:07:30.242666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.242541    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"
Jul 31 13:07:30.242666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.242551    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" event={"ID":"25ca4e54-7f86-4509-8ac9-990c6fe88945","Type":"ContainerDied","Data":"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"}
Jul 31 13:07:30.242666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.242599    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85" event={"ID":"25ca4e54-7f86-4509-8ac9-990c6fe88945","Type":"ContainerDied","Data":"fa392d98e18e2cc268dc017c4d129667584d6ec038fd282a086ccb5d6a455c95"}
Jul 31 13:07:30.242666 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.242621    2532 scope.go:117] "RemoveContainer" containerID="fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"
Jul 31 13:07:30.254897 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.254877    2532 scope.go:117] "RemoveContainer" containerID="fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"
Jul 31 13:07:30.255425 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:07:30.255393    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76\": container with ID starting with fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76 not found: ID does not exist" containerID="fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"
Jul 31 13:07:30.255523 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.255430    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76"} err="failed to get container status \"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76\": rpc error: code = NotFound desc = could not find container \"fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76\": container with ID starting with fbc515db52b70a3de82b0faeee2d8f6aba668896e0b698b81293fd58bae24a76 not found: ID does not exist"
Jul 31 13:07:30.273408 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.267626    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:07:30.273408 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.272670    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-96b3e0/auth-test-model-96b3e0-kserve-b48c68f99-q4w85"]
Jul 31 13:07:30.773416 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:30.773370    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" podUID="602f1c6f-68f3-479b-938a-0f74cb3c897a" containerName="main" containerID="cri-o://79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542" gracePeriod=45
Jul 31 13:07:31.038532 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.038474    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:07:31.202590 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202562    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202608    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xmmw\" (UniqueName: \"kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202641    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202705    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202952 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202791    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202952 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202798    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:31.202952 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202819    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home\") pod \"602f1c6f-68f3-479b-938a-0f74cb3c897a\" (UID: \"602f1c6f-68f3-479b-938a-0f74cb3c897a\") "
Jul 31 13:07:31.202952 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.202908    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache" (OuterVolumeSpecName: "model-cache") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:31.203162 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.203117    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home" (OuterVolumeSpecName: "home") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:31.203245 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.203196    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-tmp-dir\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.203245 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.203219    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-model-cache\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.204958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.204929    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:07:31.205070 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.204978    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw" (OuterVolumeSpecName: "kube-api-access-7xmmw") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "kube-api-access-7xmmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:07:31.205070 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.205035    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm" (OuterVolumeSpecName: "dshm") pod "602f1c6f-68f3-479b-938a-0f74cb3c897a" (UID: "602f1c6f-68f3-479b-938a-0f74cb3c897a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 13:07:31.248981 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.248951    2532 generic.go:356] "Generic (PLEG): container finished" podID="602f1c6f-68f3-479b-938a-0f74cb3c897a" containerID="79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542" exitCode=0
Jul 31 13:07:31.249114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.248994    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"
Jul 31 13:07:31.249114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.249039    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" event={"ID":"602f1c6f-68f3-479b-938a-0f74cb3c897a","Type":"ContainerDied","Data":"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"}
Jul 31 13:07:31.249114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.249083    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x" event={"ID":"602f1c6f-68f3-479b-938a-0f74cb3c897a","Type":"ContainerDied","Data":"d1f39957476c523e43496a8c633d7fb6491221ffa256c3d421da2e117abd78a9"}
Jul 31 13:07:31.249114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.249105    2532 scope.go:117] "RemoveContainer" containerID="79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"
Jul 31 13:07:31.258358 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.258342    2532 scope.go:117] "RemoveContainer" containerID="79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"
Jul 31 13:07:31.258610 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:07:31.258588    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542\": container with ID starting with 79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542 not found: ID does not exist" containerID="79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"
Jul 31 13:07:31.258683 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.258617    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542"} err="failed to get container status \"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542\": rpc error: code = NotFound desc = could not find container \"79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542\": container with ID starting with 79089d9bea9c8380b208f162d227a84f1c5cdacbe4f62a14b71e851a93cf3542 not found: ID does not exist"
Jul 31 13:07:31.273326 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.273301    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:07:31.278132 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.278112    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e55b67/auth-test-model-e55b67-kserve-5c9dbbdd75-v4t4x"]
Jul 31 13:07:31.304611 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.304558    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xmmw\" (UniqueName: \"kubernetes.io/projected/602f1c6f-68f3-479b-938a-0f74cb3c897a-kube-api-access-7xmmw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.304611 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.304581    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/602f1c6f-68f3-479b-938a-0f74cb3c897a-tls-certs\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.304611 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.304593    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-dshm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.304611 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.304600    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/602f1c6f-68f3-479b-938a-0f74cb3c897a-home\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:07:31.346833 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.346807    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ca4e54-7f86-4509-8ac9-990c6fe88945" path="/var/lib/kubelet/pods/25ca4e54-7f86-4509-8ac9-990c6fe88945/volumes"
Jul 31 13:07:31.347234 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:31.347217    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="602f1c6f-68f3-479b-938a-0f74cb3c897a" path="/var/lib/kubelet/pods/602f1c6f-68f3-479b-938a-0f74cb3c897a/volumes"
Jul 31 13:07:59.957158 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:07:59.957079    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:08:38.551944 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:08:38.551901    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:08:51.460184 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:08:51.460145    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:09:03.259733 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:09:03.259692    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:09:12.855351 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:09:12.855253    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:09:32.455262 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:09:32.455221    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:10:14.655878 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:10:14.655836    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:11:03.050582 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:03.050486    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:11:15.252798 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:15.252755    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:11:42.654432 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:42.654388    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:11:50.386661 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.386625    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729871792"]
Jul 31 13:11:50.387048 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387036    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ca4e54-7f86-4509-8ac9-990c6fe88945" containerName="main"
Jul 31 13:11:50.387106 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387050    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ca4e54-7f86-4509-8ac9-990c6fe88945" containerName="main"
Jul 31 13:11:50.387106 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387060    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602f1c6f-68f3-479b-938a-0f74cb3c897a" containerName="main"
Jul 31 13:11:50.387106 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387065    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="602f1c6f-68f3-479b-938a-0f74cb3c897a" containerName="main"
Jul 31 13:11:50.387227 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387142    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="602f1c6f-68f3-479b-938a-0f74cb3c897a" containerName="main"
Jul 31 13:11:50.387227 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.387152    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ca4e54-7f86-4509-8ac9-990c6fe88945" containerName="main"
Jul 31 13:11:50.390223 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.390205    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:50.392523 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.392495    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-phs9w\""
Jul 31 13:11:50.399785 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.399757    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729871792"]
Jul 31 13:11:50.507731 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.507704    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95wb\" (UniqueName: \"kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb\") pod \"test-curl-7724-140524729871792\" (UID: \"d4709e5c-2cbe-4252-a228-4e3fcef4f395\") " pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:50.608834 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.608801    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95wb\" (UniqueName: \"kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb\") pod \"test-curl-7724-140524729871792\" (UID: \"d4709e5c-2cbe-4252-a228-4e3fcef4f395\") " pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:50.618409 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.618382    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95wb\" (UniqueName: \"kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb\") pod \"test-curl-7724-140524729871792\" (UID: \"d4709e5c-2cbe-4252-a228-4e3fcef4f395\") " pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:50.700648 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.700563    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:50.837638 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.837615    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729871792"]
Jul 31 13:11:50.839833 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:11:50.839801    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4709e5c_2cbe_4252_a228_4e3fcef4f395.slice/crio-6c037f018005cb1a7040e2dc43b8c4d11eddc0c240309ce4a7cdf671f5ee07a8 WatchSource:0}: Error finding container 6c037f018005cb1a7040e2dc43b8c4d11eddc0c240309ce4a7cdf671f5ee07a8: Status 404 returned error can't find the container with id 6c037f018005cb1a7040e2dc43b8c4d11eddc0c240309ce4a7cdf671f5ee07a8
Jul 31 13:11:50.841354 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:50.841333    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 13:11:51.730326 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:51.730295    2532 generic.go:356] "Generic (PLEG): container finished" podID="d4709e5c-2cbe-4252-a228-4e3fcef4f395" containerID="3015f3613950500008cbf722b5d0b20f761dd49f09f7d55c8f2d790bc3e1e9f4" exitCode=0
Jul 31 13:11:51.730648 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:51.730344    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729871792" event={"ID":"d4709e5c-2cbe-4252-a228-4e3fcef4f395","Type":"ContainerDied","Data":"3015f3613950500008cbf722b5d0b20f761dd49f09f7d55c8f2d790bc3e1e9f4"}
Jul 31 13:11:51.730648 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:51.730384    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729871792" event={"ID":"d4709e5c-2cbe-4252-a228-4e3fcef4f395","Type":"ContainerStarted","Data":"6c037f018005cb1a7040e2dc43b8c4d11eddc0c240309ce4a7cdf671f5ee07a8"}
Jul 31 13:11:52.868590 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:52.868565    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:52.904911 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:52.904881    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729871792_d4709e5c-2cbe-4252-a228-4e3fcef4f395/test-curl-7724-140524729871792/0.log"
Jul 31 13:11:52.931406 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:52.931374    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729871792"]
Jul 31 13:11:52.933615 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:52.933592    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729871792"]
Jul 31 13:11:53.031079 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.031006    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c95wb\" (UniqueName: \"kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb\") pod \"d4709e5c-2cbe-4252-a228-4e3fcef4f395\" (UID: \"d4709e5c-2cbe-4252-a228-4e3fcef4f395\") "
Jul 31 13:11:53.033123 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.033093    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb" (OuterVolumeSpecName: "kube-api-access-c95wb") pod "d4709e5c-2cbe-4252-a228-4e3fcef4f395" (UID: "d4709e5c-2cbe-4252-a228-4e3fcef4f395"). InnerVolumeSpecName "kube-api-access-c95wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:11:53.059006 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.058979    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729872048"]
Jul 31 13:11:53.059428 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.059412    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4709e5c-2cbe-4252-a228-4e3fcef4f395" containerName="test-curl-7724-140524729871792"
Jul 31 13:11:53.059482 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.059430    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4709e5c-2cbe-4252-a228-4e3fcef4f395" containerName="test-curl-7724-140524729871792"
Jul 31 13:11:53.059518 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.059498    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4709e5c-2cbe-4252-a228-4e3fcef4f395" containerName="test-curl-7724-140524729871792"
Jul 31 13:11:53.062464 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.062446    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:53.070648 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.070627    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729872048"]
Jul 31 13:11:53.131646 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.131618    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95wb\" (UniqueName: \"kubernetes.io/projected/d4709e5c-2cbe-4252-a228-4e3fcef4f395-kube-api-access-c95wb\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:11:53.232776 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.232743    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sl4b\" (UniqueName: \"kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b\") pod \"test-curl-7724-140524729872048\" (UID: \"6cbbe87b-8669-43c9-bbd6-238760014760\") " pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:53.333660 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.333578    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sl4b\" (UniqueName: \"kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b\") pod \"test-curl-7724-140524729872048\" (UID: \"6cbbe87b-8669-43c9-bbd6-238760014760\") " pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:53.342636 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.342607    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sl4b\" (UniqueName: \"kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b\") pod \"test-curl-7724-140524729872048\" (UID: \"6cbbe87b-8669-43c9-bbd6-238760014760\") " pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:53.356211 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.356155    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4709e5c-2cbe-4252-a228-4e3fcef4f395" path="/var/lib/kubelet/pods/d4709e5c-2cbe-4252-a228-4e3fcef4f395/volumes"
Jul 31 13:11:53.371873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.371850    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:53.508310 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.508145    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729872048"]
Jul 31 13:11:53.510633 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:11:53.510600    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cbbe87b_8669_43c9_bbd6_238760014760.slice/crio-475b12db1631e700490e2dac27812f85c8e7ffc388e119ee5ec5f0f4584f32b2 WatchSource:0}: Error finding container 475b12db1631e700490e2dac27812f85c8e7ffc388e119ee5ec5f0f4584f32b2: Status 404 returned error can't find the container with id 475b12db1631e700490e2dac27812f85c8e7ffc388e119ee5ec5f0f4584f32b2
Jul 31 13:11:53.743230 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.743190    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729872048" event={"ID":"6cbbe87b-8669-43c9-bbd6-238760014760","Type":"ContainerStarted","Data":"475b12db1631e700490e2dac27812f85c8e7ffc388e119ee5ec5f0f4584f32b2"}
Jul 31 13:11:53.744444 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.744424    2532 scope.go:117] "RemoveContainer" containerID="3015f3613950500008cbf722b5d0b20f761dd49f09f7d55c8f2d790bc3e1e9f4"
Jul 31 13:11:53.744566 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:53.744451    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729871792"
Jul 31 13:11:54.062885 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:54.062818    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:11:54.751644 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:54.751604    2532 generic.go:356] "Generic (PLEG): container finished" podID="6cbbe87b-8669-43c9-bbd6-238760014760" containerID="bd4f2eb0e4709b402288cf3d33325b089add4ae49120be34448117e2f067e848" exitCode=0
Jul 31 13:11:54.751823 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:54.751688    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729872048" event={"ID":"6cbbe87b-8669-43c9-bbd6-238760014760","Type":"ContainerDied","Data":"bd4f2eb0e4709b402288cf3d33325b089add4ae49120be34448117e2f067e848"}
Jul 31 13:11:55.892404 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.892381    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:55.911634 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.911609    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729872048_6cbbe87b-8669-43c9-bbd6-238760014760/test-curl-7724-140524729872048/0.log"
Jul 31 13:11:55.933900 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.933866    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729872048"]
Jul 31 13:11:55.937464 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.937432    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729872048"]
Jul 31 13:11:55.962128 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.962102    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sl4b\" (UniqueName: \"kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b\") pod \"6cbbe87b-8669-43c9-bbd6-238760014760\" (UID: \"6cbbe87b-8669-43c9-bbd6-238760014760\") "
Jul 31 13:11:55.964085 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:55.964059    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b" (OuterVolumeSpecName: "kube-api-access-4sl4b") pod "6cbbe87b-8669-43c9-bbd6-238760014760" (UID: "6cbbe87b-8669-43c9-bbd6-238760014760"). InnerVolumeSpecName "kube-api-access-4sl4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:11:56.062968 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.062907    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sl4b\" (UniqueName: \"kubernetes.io/projected/6cbbe87b-8669-43c9-bbd6-238760014760-kube-api-access-4sl4b\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:11:56.095894 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.095871    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:56.096286 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.096274    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbbe87b-8669-43c9-bbd6-238760014760" containerName="test-curl-7724-140524729872048"
Jul 31 13:11:56.096334 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.096288    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbbe87b-8669-43c9-bbd6-238760014760" containerName="test-curl-7724-140524729872048"
Jul 31 13:11:56.096369 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.096356    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cbbe87b-8669-43c9-bbd6-238760014760" containerName="test-curl-7724-140524729872048"
Jul 31 13:11:56.099406 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.099388    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:56.106124 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.106095    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:56.163711 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.163683    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxpnd\" (UniqueName: \"kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd\") pod \"test-curl-7724-140524729666704\" (UID: \"24249d33-c743-43ad-8c4c-56aef1b8357b\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:56.264976 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.264947    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxpnd\" (UniqueName: \"kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd\") pod \"test-curl-7724-140524729666704\" (UID: \"24249d33-c743-43ad-8c4c-56aef1b8357b\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:56.273600 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.273568    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxpnd\" (UniqueName: \"kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd\") pod \"test-curl-7724-140524729666704\" (UID: \"24249d33-c743-43ad-8c4c-56aef1b8357b\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:56.410863 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.410784    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:56.544506 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.544481    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:56.546078 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:11:56.546050    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24249d33_c743_43ad_8c4c_56aef1b8357b.slice/crio-be958ef37307577ece1bd67d091dfa1bdc5a896c84710fca01be6898ec95c25c WatchSource:0}: Error finding container be958ef37307577ece1bd67d091dfa1bdc5a896c84710fca01be6898ec95c25c: Status 404 returned error can't find the container with id be958ef37307577ece1bd67d091dfa1bdc5a896c84710fca01be6898ec95c25c
Jul 31 13:11:56.765795 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.765761    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729666704" event={"ID":"24249d33-c743-43ad-8c4c-56aef1b8357b","Type":"ContainerStarted","Data":"be958ef37307577ece1bd67d091dfa1bdc5a896c84710fca01be6898ec95c25c"}
Jul 31 13:11:56.767200 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.767165    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="475b12db1631e700490e2dac27812f85c8e7ffc388e119ee5ec5f0f4584f32b2"
Jul 31 13:11:56.767200 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:56.767187    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:56.777521 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:11:56.777495    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7724-140524729872048\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="6cbbe87b-8669-43c9-bbd6-238760014760" pod="opendatahub/test-curl-7724-140524729872048"
Jul 31 13:11:57.347509 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:57.347469    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbbe87b-8669-43c9-bbd6-238760014760" path="/var/lib/kubelet/pods/6cbbe87b-8669-43c9-bbd6-238760014760/volumes"
Jul 31 13:11:57.774706 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:57.774671    2532 generic.go:356] "Generic (PLEG): container finished" podID="24249d33-c743-43ad-8c4c-56aef1b8357b" containerID="a430b2f67d22254bdbd22bb71368ad2896d673086092b8153649d3c5b62351aa" exitCode=0
Jul 31 13:11:57.774887 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:57.774758    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729666704" event={"ID":"24249d33-c743-43ad-8c4c-56aef1b8357b","Type":"ContainerDied","Data":"a430b2f67d22254bdbd22bb71368ad2896d673086092b8153649d3c5b62351aa"}
Jul 31 13:11:58.914592 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.914567    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:58.935114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.935089    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729666704_24249d33-c743-43ad-8c4c-56aef1b8357b/test-curl-7724-140524729666704/0.log"
Jul 31 13:11:58.965588 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.965564    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:58.972187 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.972153    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:58.988418 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.988395    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxpnd\" (UniqueName: \"kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd\") pod \"24249d33-c743-43ad-8c4c-56aef1b8357b\" (UID: \"24249d33-c743-43ad-8c4c-56aef1b8357b\") "
Jul 31 13:11:58.990443 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:58.990415    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd" (OuterVolumeSpecName: "kube-api-access-qxpnd") pod "24249d33-c743-43ad-8c4c-56aef1b8357b" (UID: "24249d33-c743-43ad-8c4c-56aef1b8357b"). InnerVolumeSpecName "kube-api-access-qxpnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:11:59.087243 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.087184    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:59.087623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.087610    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24249d33-c743-43ad-8c4c-56aef1b8357b" containerName="test-curl-7724-140524729666704"
Jul 31 13:11:59.087659 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.087625    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="24249d33-c743-43ad-8c4c-56aef1b8357b" containerName="test-curl-7724-140524729666704"
Jul 31 13:11:59.087731 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.087721    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="24249d33-c743-43ad-8c4c-56aef1b8357b" containerName="test-curl-7724-140524729666704"
Jul 31 13:11:59.089935 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.089915    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxpnd\" (UniqueName: \"kubernetes.io/projected/24249d33-c743-43ad-8c4c-56aef1b8357b-kube-api-access-qxpnd\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:11:59.090850 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.090832    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.106344 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.098069    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:59.191120 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.191089    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9dl2\" (UniqueName: \"kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2\") pod \"test-curl-7724-140524729666704\" (UID: \"741cc9c7-ac81-4574-95f2-1418da97e191\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.292439 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.292408    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dl2\" (UniqueName: \"kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2\") pod \"test-curl-7724-140524729666704\" (UID: \"741cc9c7-ac81-4574-95f2-1418da97e191\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.306945 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.306914    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9dl2\" (UniqueName: \"kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2\") pod \"test-curl-7724-140524729666704\" (UID: \"741cc9c7-ac81-4574-95f2-1418da97e191\") " pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.346268 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.346196    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24249d33-c743-43ad-8c4c-56aef1b8357b" path="/var/lib/kubelet/pods/24249d33-c743-43ad-8c4c-56aef1b8357b/volumes"
Jul 31 13:11:59.414131 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.414107    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.547014 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.546984    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:11:59.548987 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:11:59.548957    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod741cc9c7_ac81_4574_95f2_1418da97e191.slice/crio-469676067f00af0d6a94e9a4190cc4ff68f49a4fc49e2c4216e1af280eed774d WatchSource:0}: Error finding container 469676067f00af0d6a94e9a4190cc4ff68f49a4fc49e2c4216e1af280eed774d: Status 404 returned error can't find the container with id 469676067f00af0d6a94e9a4190cc4ff68f49a4fc49e2c4216e1af280eed774d
Jul 31 13:11:59.788104 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.788081    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:11:59.788265 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.788117    2532 scope.go:117] "RemoveContainer" containerID="a430b2f67d22254bdbd22bb71368ad2896d673086092b8153649d3c5b62351aa"
Jul 31 13:11:59.789564 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:11:59.789533    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729666704" event={"ID":"741cc9c7-ac81-4574-95f2-1418da97e191","Type":"ContainerStarted","Data":"469676067f00af0d6a94e9a4190cc4ff68f49a4fc49e2c4216e1af280eed774d"}
Jul 31 13:12:00.797449 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:00.797413    2532 generic.go:356] "Generic (PLEG): container finished" podID="741cc9c7-ac81-4574-95f2-1418da97e191" containerID="72e3bc210978cc056ef96396366b29c632e4bbc4234ffed7e53a51d92be74d7a" exitCode=0
Jul 31 13:12:00.797865 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:00.797492    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729666704" event={"ID":"741cc9c7-ac81-4574-95f2-1418da97e191","Type":"ContainerDied","Data":"72e3bc210978cc056ef96396366b29c632e4bbc4234ffed7e53a51d92be74d7a"}
Jul 31 13:12:01.944163 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:01.944133    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:12:01.963696 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:01.963666    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729666704_741cc9c7-ac81-4574-95f2-1418da97e191/test-curl-7724-140524729666704/0.log"
Jul 31 13:12:01.987254 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:01.987228    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:12:01.990282 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:01.990257    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729666704"]
Jul 31 13:12:02.018895 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.018874    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9dl2\" (UniqueName: \"kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2\") pod \"741cc9c7-ac81-4574-95f2-1418da97e191\" (UID: \"741cc9c7-ac81-4574-95f2-1418da97e191\") "
Jul 31 13:12:02.020860 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.020840    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2" (OuterVolumeSpecName: "kube-api-access-h9dl2") pod "741cc9c7-ac81-4574-95f2-1418da97e191" (UID: "741cc9c7-ac81-4574-95f2-1418da97e191"). InnerVolumeSpecName "kube-api-access-h9dl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:12:02.106964 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.106896    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:02.107317 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.107304    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" containerName="test-curl-7724-140524729666704"
Jul 31 13:12:02.107369 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.107319    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" containerName="test-curl-7724-140524729666704"
Jul 31 13:12:02.107420 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.107411    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" containerName="test-curl-7724-140524729666704"
Jul 31 13:12:02.110349 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.110333    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:02.117029 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.117008    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:02.119543 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.119519    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9dl2\" (UniqueName: \"kubernetes.io/projected/741cc9c7-ac81-4574-95f2-1418da97e191-kube-api-access-h9dl2\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:12:02.219915 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.219885    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdsvr\" (UniqueName: \"kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr\") pod \"test-curl-7724-140524729888816\" (UID: \"55a2c07b-7031-450a-9b5d-1c5aba8f8553\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:02.321339 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.321310    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdsvr\" (UniqueName: \"kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr\") pod \"test-curl-7724-140524729888816\" (UID: \"55a2c07b-7031-450a-9b5d-1c5aba8f8553\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:02.329866 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.329819    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdsvr\" (UniqueName: \"kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr\") pod \"test-curl-7724-140524729888816\" (UID: \"55a2c07b-7031-450a-9b5d-1c5aba8f8553\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:02.420036 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.419969    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:02.561300 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.561267    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:02.562454 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:12:02.562415    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55a2c07b_7031_450a_9b5d_1c5aba8f8553.slice/crio-1b3b37e6f8eccf6fc8213feaba3e3324e812dff5977a024133f9b9a1457d87c9 WatchSource:0}: Error finding container 1b3b37e6f8eccf6fc8213feaba3e3324e812dff5977a024133f9b9a1457d87c9: Status 404 returned error can't find the container with id 1b3b37e6f8eccf6fc8213feaba3e3324e812dff5977a024133f9b9a1457d87c9
Jul 31 13:12:02.810487 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.810459    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:12:02.810639 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.810499    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="469676067f00af0d6a94e9a4190cc4ff68f49a4fc49e2c4216e1af280eed774d"
Jul 31 13:12:02.811664 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:02.811643    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729888816" event={"ID":"55a2c07b-7031-450a-9b5d-1c5aba8f8553","Type":"ContainerStarted","Data":"1b3b37e6f8eccf6fc8213feaba3e3324e812dff5977a024133f9b9a1457d87c9"}
Jul 31 13:12:02.821373 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:12:02.821350    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7724-140524729666704\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:12:03.346000 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:12:03.345967    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7724-140524729666704\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" pod="opendatahub/test-curl-7724-140524729666704"
Jul 31 13:12:03.347593 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:03.347566    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="741cc9c7-ac81-4574-95f2-1418da97e191" path="/var/lib/kubelet/pods/741cc9c7-ac81-4574-95f2-1418da97e191/volumes"
Jul 31 13:12:03.820778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:03.820742    2532 generic.go:356] "Generic (PLEG): container finished" podID="55a2c07b-7031-450a-9b5d-1c5aba8f8553" containerID="5f48837b031f9e8fec0812b0326e40c8665553806c46c747ec0646f46a058f0e" exitCode=0
Jul 31 13:12:03.820961 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:03.820788    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729888816" event={"ID":"55a2c07b-7031-450a-9b5d-1c5aba8f8553","Type":"ContainerDied","Data":"5f48837b031f9e8fec0812b0326e40c8665553806c46c747ec0646f46a058f0e"}
Jul 31 13:12:04.966783 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:04.966757    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:04.989773 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:04.989753    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729888816_55a2c07b-7031-450a-9b5d-1c5aba8f8553/test-curl-7724-140524729888816/0.log"
Jul 31 13:12:05.015987 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.015952    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:05.021197 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.021160    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:05.046276 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.046254    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdsvr\" (UniqueName: \"kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr\") pod \"55a2c07b-7031-450a-9b5d-1c5aba8f8553\" (UID: \"55a2c07b-7031-450a-9b5d-1c5aba8f8553\") "
Jul 31 13:12:05.048287 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.048266    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr" (OuterVolumeSpecName: "kube-api-access-zdsvr") pod "55a2c07b-7031-450a-9b5d-1c5aba8f8553" (UID: "55a2c07b-7031-450a-9b5d-1c5aba8f8553"). InnerVolumeSpecName "kube-api-access-zdsvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:12:05.140812 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.140745    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:05.141166 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.141151    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a2c07b-7031-450a-9b5d-1c5aba8f8553" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:05.141233 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.141181    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a2c07b-7031-450a-9b5d-1c5aba8f8553" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:05.141269 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.141262    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a2c07b-7031-450a-9b5d-1c5aba8f8553" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:05.144357 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.144341    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.147045 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.147023    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdsvr\" (UniqueName: \"kubernetes.io/projected/55a2c07b-7031-450a-9b5d-1c5aba8f8553-kube-api-access-zdsvr\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:12:05.150597 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.150573    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:05.247969 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.247894    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8km\" (UniqueName: \"kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km\") pod \"test-curl-7724-140524729888816\" (UID: \"3a6cc430-b78a-4d98-8991-2f645ec19f54\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.346952 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.346919    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a2c07b-7031-450a-9b5d-1c5aba8f8553" path="/var/lib/kubelet/pods/55a2c07b-7031-450a-9b5d-1c5aba8f8553/volumes"
Jul 31 13:12:05.348481 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.348457    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8km\" (UniqueName: \"kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km\") pod \"test-curl-7724-140524729888816\" (UID: \"3a6cc430-b78a-4d98-8991-2f645ec19f54\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.355828 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.355805    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8km\" (UniqueName: \"kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km\") pod \"test-curl-7724-140524729888816\" (UID: \"3a6cc430-b78a-4d98-8991-2f645ec19f54\") " pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.455128 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.455058    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.586611 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.586580    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:05.588645 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:12:05.588616    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a6cc430_b78a_4d98_8991_2f645ec19f54.slice/crio-6c233c79ead072500ac8c8eb35d28aa3376cbd528290780d48b26434600ac918 WatchSource:0}: Error finding container 6c233c79ead072500ac8c8eb35d28aa3376cbd528290780d48b26434600ac918: Status 404 returned error can't find the container with id 6c233c79ead072500ac8c8eb35d28aa3376cbd528290780d48b26434600ac918
Jul 31 13:12:05.835316 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.835289    2532 scope.go:117] "RemoveContainer" containerID="5f48837b031f9e8fec0812b0326e40c8665553806c46c747ec0646f46a058f0e"
Jul 31 13:12:05.835491 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.835288    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:05.836650 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:05.836622    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729888816" event={"ID":"3a6cc430-b78a-4d98-8991-2f645ec19f54","Type":"ContainerStarted","Data":"6c233c79ead072500ac8c8eb35d28aa3376cbd528290780d48b26434600ac918"}
Jul 31 13:12:06.845001 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:06.844968    2532 generic.go:356] "Generic (PLEG): container finished" podID="3a6cc430-b78a-4d98-8991-2f645ec19f54" containerID="8983e42b15d31ae4fd6e14e007adbba1f5e2364fdedd90e8aba7588e62906edd" exitCode=0
Jul 31 13:12:06.845415 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:06.845057    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7724-140524729888816" event={"ID":"3a6cc430-b78a-4d98-8991-2f645ec19f54","Type":"ContainerDied","Data":"8983e42b15d31ae4fd6e14e007adbba1f5e2364fdedd90e8aba7588e62906edd"}
Jul 31 13:12:07.989965 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:07.989939    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:08.010384 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.010353    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7724-140524729888816_3a6cc430-b78a-4d98-8991-2f645ec19f54/test-curl-7724-140524729888816/0.log"
Jul 31 13:12:08.032116 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.032091    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:08.035582 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.035562    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7724-140524729888816"]
Jul 31 13:12:08.074625 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.074602    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj8km\" (UniqueName: \"kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km\") pod \"3a6cc430-b78a-4d98-8991-2f645ec19f54\" (UID: \"3a6cc430-b78a-4d98-8991-2f645ec19f54\") "
Jul 31 13:12:08.076632 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.076608    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km" (OuterVolumeSpecName: "kube-api-access-dj8km") pod "3a6cc430-b78a-4d98-8991-2f645ec19f54" (UID: "3a6cc430-b78a-4d98-8991-2f645ec19f54"). InnerVolumeSpecName "kube-api-access-dj8km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:12:08.175613 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.175523    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj8km\" (UniqueName: \"kubernetes.io/projected/3a6cc430-b78a-4d98-8991-2f645ec19f54-kube-api-access-dj8km\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:12:08.865791 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.865763    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c233c79ead072500ac8c8eb35d28aa3376cbd528290780d48b26434600ac918"
Jul 31 13:12:08.865791 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:08.865794    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:08.876265 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:12:08.876236    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7724-140524729888816\" is forbidden: User \"system:node:ip-10-0-141-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-165.ec2.internal' and this object" podUID="3a6cc430-b78a-4d98-8991-2f645ec19f54" pod="opendatahub/test-curl-7724-140524729888816"
Jul 31 13:12:09.346629 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:09.346595    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a6cc430-b78a-4d98-8991-2f645ec19f54" path="/var/lib/kubelet/pods/3a6cc430-b78a-4d98-8991-2f645ec19f54/volumes"
Jul 31 13:12:14.685988 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.685949    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:12:14.686436 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.686420    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6cc430-b78a-4d98-8991-2f645ec19f54" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:14.686481 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.686438    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6cc430-b78a-4d98-8991-2f645ec19f54" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:14.686516 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.686499    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6cc430-b78a-4d98-8991-2f645ec19f54" containerName="test-curl-7724-140524729888816"
Jul 31 13:12:14.690950 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.690932    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.693335 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.693314    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 13:12:14.694117 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.694094    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 31 13:12:14.694266 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.694118    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-l5fx5\""
Jul 31 13:12:14.694266 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.694099    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-fd6ea7-serving-cert\""
Jul 31 13:12:14.700588 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.700561    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:12:14.730962 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.730935    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.731084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.730997    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n4zf\" (UniqueName: \"kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.731084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.731029    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.831849 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.831819    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n4zf\" (UniqueName: \"kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.831962 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.831857    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.832022 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.831961    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.832075 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:12:14.832020    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-serving-cert: secret "maas-api-e2e-ipp-a-fd6ea7-serving-cert" not found
Jul 31 13:12:14.832114 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:12:14.832095    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls podName:f13c777e-e991-4e35-8652-1c22671baf27 nodeName:}" failed. No retries permitted until 2026-07-31 13:12:15.332068731 +0000 UTC m=+3422.563933462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls") pod "maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" (UID: "f13c777e-e991-4e35-8652-1c22671baf27") : secret "maas-api-e2e-ipp-a-fd6ea7-serving-cert" not found
Jul 31 13:12:14.832623 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.832607    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:14.840302 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:14.840274    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n4zf\" (UniqueName: \"kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:15.336566 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:15.336530    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:15.338915 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:15.338880    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:15.604445 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:15.604362    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:15.950213 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:15.950188    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:12:15.952081 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:12:15.952052    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13c777e_e991_4e35_8652_1c22671baf27.slice/crio-61d3a99eb00a2518c704cdcc5f693ee2480b7c9098abb6ed5f152458338aa73e WatchSource:0}: Error finding container 61d3a99eb00a2518c704cdcc5f693ee2480b7c9098abb6ed5f152458338aa73e: Status 404 returned error can't find the container with id 61d3a99eb00a2518c704cdcc5f693ee2480b7c9098abb6ed5f152458338aa73e
Jul 31 13:12:16.917021 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:16.916981    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" event={"ID":"f13c777e-e991-4e35-8652-1c22671baf27","Type":"ContainerStarted","Data":"61d3a99eb00a2518c704cdcc5f693ee2480b7c9098abb6ed5f152458338aa73e"}
Jul 31 13:12:17.662994 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:17.662961    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:12:18.938408 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:18.938371    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" event={"ID":"f13c777e-e991-4e35-8652-1c22671baf27","Type":"ContainerStarted","Data":"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"}
Jul 31 13:12:18.938890 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:18.938455    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:18.955493 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:18.955442    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" podStartSLOduration=2.661712615 podStartE2EDuration="4.95542936s" totalImagesPullingTime="2.293716745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:12:18.953197896 +0000 UTC m=+3426.185062641" watchObservedRunningTime="2026-07-31 13:12:18.95542936 +0000 UTC m=+3426.187294110"
Jul 31 13:12:24.949880 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:24.949830    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:12:37.832342 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.832309    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:12:37.836178 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.836144    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:37.838468 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.838449    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f59489-serving-cert\""
Jul 31 13:12:37.846718 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.846697    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:12:37.942640 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.942613    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:37.942782 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.942715    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:37.942830 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:37.942784    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szfgw\" (UniqueName: \"kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.043416 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.043390    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szfgw\" (UniqueName: \"kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.043566 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.043444    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.043566 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.043490    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.044237 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.044212    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.045980 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.045959    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.051999 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.051976    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szfgw\" (UniqueName: \"kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw\") pod \"maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.148774 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.148702    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:38.295731 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.295703    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:12:38.298320 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:12:38.298279    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3510203d_81fb_46f4_8af2_c13a99541503.slice/crio-f81934af59144ae54111bb5786c5638e26adc410d061d014d40b7a7dacb51c79 WatchSource:0}: Error finding container f81934af59144ae54111bb5786c5638e26adc410d061d014d40b7a7dacb51c79: Status 404 returned error can't find the container with id f81934af59144ae54111bb5786c5638e26adc410d061d014d40b7a7dacb51c79
Jul 31 13:12:38.418565 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.418490    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:12:38.423327 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.423303    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.425626 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.425606    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f59489-dockercfg-lpz68\""
Jul 31 13:12:38.425719 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.425673    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f59489\""
Jul 31 13:12:38.433500 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.433474    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:12:38.549264 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.549234    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.549428 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.549283    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd562\" (UniqueName: \"kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.649975 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.649941    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.650111 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.649998    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd562\" (UniqueName: \"kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.650723 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.650703    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.657439 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.657418    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd562\" (UniqueName: \"kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562\") pod \"payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.737939 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.737913    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:38.869787 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:38.869760    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:12:38.871919 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:12:38.871887    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916454d0_0479_4a97_8d56_04012cba80b5.slice/crio-8bb42e6a06e3b6034e8afa7a21d83c24f78bc349d6a6987ed11e0911abd9c630 WatchSource:0}: Error finding container 8bb42e6a06e3b6034e8afa7a21d83c24f78bc349d6a6987ed11e0911abd9c630: Status 404 returned error can't find the container with id 8bb42e6a06e3b6034e8afa7a21d83c24f78bc349d6a6987ed11e0911abd9c630
Jul 31 13:12:39.063275 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.063233    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" event={"ID":"3510203d-81fb-46f4-8af2-c13a99541503","Type":"ContainerStarted","Data":"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"}
Jul 31 13:12:39.063427 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.063406    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:39.063496 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.063456    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" event={"ID":"3510203d-81fb-46f4-8af2-c13a99541503","Type":"ContainerStarted","Data":"f81934af59144ae54111bb5786c5638e26adc410d061d014d40b7a7dacb51c79"}
Jul 31 13:12:39.064872 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.064846    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" event={"ID":"916454d0-0479-4a97-8d56-04012cba80b5","Type":"ContainerStarted","Data":"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"}
Jul 31 13:12:39.064971 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.064888    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" event={"ID":"916454d0-0479-4a97-8d56-04012cba80b5","Type":"ContainerStarted","Data":"8bb42e6a06e3b6034e8afa7a21d83c24f78bc349d6a6987ed11e0911abd9c630"}
Jul 31 13:12:39.064971 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.064915    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:12:39.080857 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.080812    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" podStartSLOduration=1.728003535 podStartE2EDuration="2.080799428s" totalImagesPullingTime="352.795893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:12:39.077538669 +0000 UTC m=+3446.309403443" watchObservedRunningTime="2026-07-31 13:12:39.080799428 +0000 UTC m=+3446.312664228"
Jul 31 13:12:39.092513 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:39.092475    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" podStartSLOduration=1.092463741 podStartE2EDuration="1.092463741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:12:39.091879433 +0000 UTC m=+3446.323744186" watchObservedRunningTime="2026-07-31 13:12:39.092463741 +0000 UTC m=+3446.324328492"
Jul 31 13:12:40.655605 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:40.655570    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:12:45.077489 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:45.077454    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:12:50.072808 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:12:50.072775    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:13:12.049060 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:12.049015    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:13:36.649846 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:36.649750    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:13:53.999364 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:53.999324    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:13:54.003112 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.003088    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.005290 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.005258    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2b382a-serving-cert\""
Jul 31 13:13:54.018409 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.018376    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:13:54.070700 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.070657    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws2xc\" (UniqueName: \"kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.070829 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.070719    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.070885 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.070861    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.171998 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.171961    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.172196 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.172007    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws2xc\" (UniqueName: \"kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.172196 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:13:54.172130    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-serving-cert: secret "maas-api-e2e-ipp-cleanup-2b382a-serving-cert" not found
Jul 31 13:13:54.172318 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:13:54.172243    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls podName:7cc44dc5-bb6f-4a41-8a17-a8422216a906 nodeName:}" failed. No retries permitted until 2026-07-31 13:13:54.672220824 +0000 UTC m=+3521.904085555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" (UID: "7cc44dc5-bb6f-4a41-8a17-a8422216a906") : secret "maas-api-e2e-ipp-cleanup-2b382a-serving-cert" not found
Jul 31 13:13:54.172318 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.172133    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.172796 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.172778    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.180136 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.180112    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws2xc\" (UniqueName: \"kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.615538 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.615506    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:13:54.619393 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.619377    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.621678 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.621658    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-2b382a\""
Jul 31 13:13:54.621824 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.621799    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-2b382a-dockercfg-rzqzj\""
Jul 31 13:13:54.626476 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.626457    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:13:54.677798 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.677764    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.680187 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.680138    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.779136 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.779100    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsllp\" (UniqueName: \"kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.779312 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.779190    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.880401 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.880322    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.880536 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.880410    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsllp\" (UniqueName: \"kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.881030 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.881010    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.888007 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.887974    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsllp\" (UniqueName: \"kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp\") pod \"payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:54.926842 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.926818    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:13:54.932880 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:54.932860    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:55.080890 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.080775    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:13:55.084748 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:13:55.084714    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc44dc5_bb6f_4a41_8a17_a8422216a906.slice/crio-36f9b9867f36890a2801e3134d7138f7adc6b9eb7ea492c20b57d9c72ed746c2 WatchSource:0}: Error finding container 36f9b9867f36890a2801e3134d7138f7adc6b9eb7ea492c20b57d9c72ed746c2: Status 404 returned error can't find the container with id 36f9b9867f36890a2801e3134d7138f7adc6b9eb7ea492c20b57d9c72ed746c2
Jul 31 13:13:55.312703 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.312678    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:13:55.314346 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:13:55.314310    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45a84bd_e9c7_42f9_a59a_17114505538b.slice/crio-37d65e20f33d21721d908249fc269a84e07bb6d519e1294336c9621f5f609170 WatchSource:0}: Error finding container 37d65e20f33d21721d908249fc269a84e07bb6d519e1294336c9621f5f609170: Status 404 returned error can't find the container with id 37d65e20f33d21721d908249fc269a84e07bb6d519e1294336c9621f5f609170
Jul 31 13:13:55.507557 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.507528    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" event={"ID":"7cc44dc5-bb6f-4a41-8a17-a8422216a906","Type":"ContainerStarted","Data":"36f9b9867f36890a2801e3134d7138f7adc6b9eb7ea492c20b57d9c72ed746c2"}
Jul 31 13:13:55.509039 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.509017    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" event={"ID":"b45a84bd-e9c7-42f9-a59a-17114505538b","Type":"ContainerStarted","Data":"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"}
Jul 31 13:13:55.509150 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.509063    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" event={"ID":"b45a84bd-e9c7-42f9-a59a-17114505538b","Type":"ContainerStarted","Data":"37d65e20f33d21721d908249fc269a84e07bb6d519e1294336c9621f5f609170"}
Jul 31 13:13:55.509246 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.509158    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:13:55.525690 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:55.525643    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" podStartSLOduration=1.525629061 podStartE2EDuration="1.525629061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:13:55.523517544 +0000 UTC m=+3522.755382295" watchObservedRunningTime="2026-07-31 13:13:55.525629061 +0000 UTC m=+3522.757493790"
Jul 31 13:13:56.516834 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:56.516791    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" event={"ID":"7cc44dc5-bb6f-4a41-8a17-a8422216a906","Type":"ContainerStarted","Data":"241fd139e1b374e00f5591bf39d8625df6d3e09828b26f17bcb0ce8d64826123"}
Jul 31 13:13:56.533918 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:56.533852    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" podStartSLOduration=3.145916625 podStartE2EDuration="3.533838574s" totalImagesPullingTime="387.921949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 13:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:13:56.532124318 +0000 UTC m=+3523.763989072" watchObservedRunningTime="2026-07-31 13:13:56.533838574 +0000 UTC m=+3523.765703362"
Jul 31 13:13:57.355868 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:57.355833    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:13:57.522806 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:13:57.522778    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:14:03.534500 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:03.534472    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:14:06.519463 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:06.519433    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:14:21.516089 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.516055    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:14:21.516680 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.516257    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" podUID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" containerName="maas-api" containerID="cri-o://241fd139e1b374e00f5591bf39d8625df6d3e09828b26f17bcb0ce8d64826123" gracePeriod=30
Jul 31 13:14:21.560607 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.560579    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:14:21.560815 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.560788    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" podUID="b45a84bd-e9c7-42f9-a59a-17114505538b" containerName="payload-processing" containerID="cri-o://19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535" gracePeriod=30
Jul 31 13:14:21.661612 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.661584    2532 generic.go:356] "Generic (PLEG): container finished" podID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" containerID="241fd139e1b374e00f5591bf39d8625df6d3e09828b26f17bcb0ce8d64826123" exitCode=0
Jul 31 13:14:21.661750 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.661662    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" event={"ID":"7cc44dc5-bb6f-4a41-8a17-a8422216a906","Type":"ContainerDied","Data":"241fd139e1b374e00f5591bf39d8625df6d3e09828b26f17bcb0ce8d64826123"}
Jul 31 13:14:21.818515 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.818491    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:14:21.838845 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.838823    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:14:21.911091 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911046    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") pod \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") "
Jul 31 13:14:21.911242 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911112    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws2xc\" (UniqueName: \"kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc\") pod \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") "
Jul 31 13:14:21.911242 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911237    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca\") pod \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\" (UID: \"7cc44dc5-bb6f-4a41-8a17-a8422216a906\") "
Jul 31 13:14:21.911344 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911256    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume\") pod \"b45a84bd-e9c7-42f9-a59a-17114505538b\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") "
Jul 31 13:14:21.911344 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911282    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsllp\" (UniqueName: \"kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp\") pod \"b45a84bd-e9c7-42f9-a59a-17114505538b\" (UID: \"b45a84bd-e9c7-42f9-a59a-17114505538b\") "
Jul 31 13:14:21.911600 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911570    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7cc44dc5-bb6f-4a41-8a17-a8422216a906" (UID: "7cc44dc5-bb6f-4a41-8a17-a8422216a906"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:14:21.911681 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.911646    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b45a84bd-e9c7-42f9-a59a-17114505538b" (UID: "b45a84bd-e9c7-42f9-a59a-17114505538b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:14:21.913307 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.913283    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp" (OuterVolumeSpecName: "kube-api-access-dsllp") pod "b45a84bd-e9c7-42f9-a59a-17114505538b" (UID: "b45a84bd-e9c7-42f9-a59a-17114505538b"). InnerVolumeSpecName "kube-api-access-dsllp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:14:21.913745 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.913724    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7cc44dc5-bb6f-4a41-8a17-a8422216a906" (UID: "7cc44dc5-bb6f-4a41-8a17-a8422216a906"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:14:21.913817 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:21.913725    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc" (OuterVolumeSpecName: "kube-api-access-ws2xc") pod "7cc44dc5-bb6f-4a41-8a17-a8422216a906" (UID: "7cc44dc5-bb6f-4a41-8a17-a8422216a906"). InnerVolumeSpecName "kube-api-access-ws2xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:14:22.012429 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.012402    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc44dc5-bb6f-4a41-8a17-a8422216a906-openshift-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:22.012429 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.012427    2532 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b45a84bd-e9c7-42f9-a59a-17114505538b-plugins-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:22.012591 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.012438    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsllp\" (UniqueName: \"kubernetes.io/projected/b45a84bd-e9c7-42f9-a59a-17114505538b-kube-api-access-dsllp\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:22.012591 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.012447    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc44dc5-bb6f-4a41-8a17-a8422216a906-maas-api-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:22.012591 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.012456    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws2xc\" (UniqueName: \"kubernetes.io/projected/7cc44dc5-bb6f-4a41-8a17-a8422216a906-kube-api-access-ws2xc\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:22.668581 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.668538    2532 generic.go:356] "Generic (PLEG): container finished" podID="b45a84bd-e9c7-42f9-a59a-17114505538b" containerID="19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535" exitCode=0
Jul 31 13:14:22.669019 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.668600    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"
Jul 31 13:14:22.669019 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.668627    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" event={"ID":"b45a84bd-e9c7-42f9-a59a-17114505538b","Type":"ContainerDied","Data":"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"}
Jul 31 13:14:22.669019 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.668715    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn" event={"ID":"b45a84bd-e9c7-42f9-a59a-17114505538b","Type":"ContainerDied","Data":"37d65e20f33d21721d908249fc269a84e07bb6d519e1294336c9621f5f609170"}
Jul 31 13:14:22.669019 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.668746    2532 scope.go:117] "RemoveContainer" containerID="19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"
Jul 31 13:14:22.670212 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.670192    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"
Jul 31 13:14:22.670317 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.670144    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr" event={"ID":"7cc44dc5-bb6f-4a41-8a17-a8422216a906","Type":"ContainerDied","Data":"36f9b9867f36890a2801e3134d7138f7adc6b9eb7ea492c20b57d9c72ed746c2"}
Jul 31 13:14:22.679304 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.679284    2532 scope.go:117] "RemoveContainer" containerID="19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"
Jul 31 13:14:22.679562 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:14:22.679540    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535\": container with ID starting with 19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535 not found: ID does not exist" containerID="19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"
Jul 31 13:14:22.679638 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.679571    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535"} err="failed to get container status \"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535\": rpc error: code = NotFound desc = could not find container \"19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535\": container with ID starting with 19d281834e000a59e7c2ba73d90448b097982e1d8461a2086d4216e24b540535 not found: ID does not exist"
Jul 31 13:14:22.679638 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.679592    2532 scope.go:117] "RemoveContainer" containerID="241fd139e1b374e00f5591bf39d8625df6d3e09828b26f17bcb0ce8d64826123"
Jul 31 13:14:22.691966 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.691944    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:14:22.697496 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.697475    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2b382a-649469cd76-w5wcn"]
Jul 31 13:14:22.705926 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.705906    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:14:22.709956 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:22.709935    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2b382a-89f845dd5-l6shr"]
Jul 31 13:14:23.346944 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:23.346911    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" path="/var/lib/kubelet/pods/7cc44dc5-bb6f-4a41-8a17-a8422216a906/volumes"
Jul 31 13:14:23.347309 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:23.347296    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b45a84bd-e9c7-42f9-a59a-17114505538b" path="/var/lib/kubelet/pods/b45a84bd-e9c7-42f9-a59a-17114505538b/volumes"
Jul 31 13:14:24.653590 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:24.653556    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:14:32.079446 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079410    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"]
Jul 31 13:14:32.079834 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079821    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b45a84bd-e9c7-42f9-a59a-17114505538b" containerName="payload-processing"
Jul 31 13:14:32.079873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079835    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b45a84bd-e9c7-42f9-a59a-17114505538b" containerName="payload-processing"
Jul 31 13:14:32.079873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079861    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" containerName="maas-api"
Jul 31 13:14:32.079873 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079867    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" containerName="maas-api"
Jul 31 13:14:32.079958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079953    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc44dc5-bb6f-4a41-8a17-a8422216a906" containerName="maas-api"
Jul 31 13:14:32.079991 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.079962    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b45a84bd-e9c7-42f9-a59a-17114505538b" containerName="payload-processing"
Jul 31 13:14:32.083081 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.083065    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:32.085310 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.085293    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxtlm\""
Jul 31 13:14:32.090410 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.090385    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"]
Jul 31 13:14:32.201240 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.201206    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n\" (UID: \"8373fde2-b6ea-47e6-81c8-c0e668dcd67e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:32.301967 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.301903    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n\" (UID: \"8373fde2-b6ea-47e6-81c8-c0e668dcd67e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:32.302678 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.302651    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n\" (UID: \"8373fde2-b6ea-47e6-81c8-c0e668dcd67e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:32.396225 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.396130    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:32.535856 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.535830    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"]
Jul 31 13:14:32.538627 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:14:32.538600    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8373fde2_b6ea_47e6_81c8_c0e668dcd67e.slice/crio-f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540 WatchSource:0}: Error finding container f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540: Status 404 returned error can't find the container with id f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540
Jul 31 13:14:32.739063 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:32.739014    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n" event={"ID":"8373fde2-b6ea-47e6-81c8-c0e668dcd67e","Type":"ContainerStarted","Data":"f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540"}
Jul 31 13:14:33.746696 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:33.746662    2532 generic.go:356] "Generic (PLEG): container finished" podID="8373fde2-b6ea-47e6-81c8-c0e668dcd67e" containerID="d6ba69463090d8bdf202ebfd8137f48cbea7907ca9f8633a40142c9ccc6d2ed9" exitCode=0
Jul 31 13:14:33.747196 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:33.746760    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n" event={"ID":"8373fde2-b6ea-47e6-81c8-c0e668dcd67e","Type":"ContainerDied","Data":"d6ba69463090d8bdf202ebfd8137f48cbea7907ca9f8633a40142c9ccc6d2ed9"}
Jul 31 13:14:34.894305 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:34.894275    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:35.028242 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.028131    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca\") pod \"8373fde2-b6ea-47e6-81c8-c0e668dcd67e\" (UID: \"8373fde2-b6ea-47e6-81c8-c0e668dcd67e\") "
Jul 31 13:14:35.028473 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.028452    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8373fde2-b6ea-47e6-81c8-c0e668dcd67e" (UID: "8373fde2-b6ea-47e6-81c8-c0e668dcd67e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:14:35.028555 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.028542    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8373fde2-b6ea-47e6-81c8-c0e668dcd67e-maas-api-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:35.760704 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.760676    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"
Jul 31 13:14:35.760704 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.760694    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n" event={"ID":"8373fde2-b6ea-47e6-81c8-c0e668dcd67e","Type":"ContainerDied","Data":"f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540"}
Jul 31 13:14:35.760913 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:35.760743    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7c9d7ccbf493e19ffc2a34d4fbded232015d13a401e8b16c5c52aeaf8157540"
Jul 31 13:14:45.049185 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:45.049135    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:14:49.265779 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.265748    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:14:49.266288 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.266015    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" podUID="f13c777e-e991-4e35-8652-1c22671baf27" containerName="maas-api" containerID="cri-o://054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19" gracePeriod=30
Jul 31 13:14:49.452688 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.452656    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"]
Jul 31 13:14:49.455477 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.455449    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-fd6ea7-zg55n"]
Jul 31 13:14:49.527795 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.527743    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:14:49.658797 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.658769    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n4zf\" (UniqueName: \"kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf\") pod \"f13c777e-e991-4e35-8652-1c22671baf27\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") "
Jul 31 13:14:49.658964 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.658850    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca\") pod \"f13c777e-e991-4e35-8652-1c22671baf27\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") "
Jul 31 13:14:49.658964 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.658886    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") pod \"f13c777e-e991-4e35-8652-1c22671baf27\" (UID: \"f13c777e-e991-4e35-8652-1c22671baf27\") "
Jul 31 13:14:49.659228 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.659165    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f13c777e-e991-4e35-8652-1c22671baf27" (UID: "f13c777e-e991-4e35-8652-1c22671baf27"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:14:49.660904 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.660879    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf" (OuterVolumeSpecName: "kube-api-access-2n4zf") pod "f13c777e-e991-4e35-8652-1c22671baf27" (UID: "f13c777e-e991-4e35-8652-1c22671baf27"). InnerVolumeSpecName "kube-api-access-2n4zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:14:49.661000 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.660962    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f13c777e-e991-4e35-8652-1c22671baf27" (UID: "f13c777e-e991-4e35-8652-1c22671baf27"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:14:49.760183 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.760153    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n4zf\" (UniqueName: \"kubernetes.io/projected/f13c777e-e991-4e35-8652-1c22671baf27-kube-api-access-2n4zf\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:49.760308 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.760205    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13c777e-e991-4e35-8652-1c22671baf27-openshift-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:49.760308 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.760221    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13c777e-e991-4e35-8652-1c22671baf27-maas-api-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:14:49.844574 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.844510    2532 generic.go:356] "Generic (PLEG): container finished" podID="f13c777e-e991-4e35-8652-1c22671baf27" containerID="054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19" exitCode=0
Jul 31 13:14:49.844574 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.844554    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"
Jul 31 13:14:49.844763 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.844601    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" event={"ID":"f13c777e-e991-4e35-8652-1c22671baf27","Type":"ContainerDied","Data":"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"}
Jul 31 13:14:49.844763 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.844658    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv" event={"ID":"f13c777e-e991-4e35-8652-1c22671baf27","Type":"ContainerDied","Data":"61d3a99eb00a2518c704cdcc5f693ee2480b7c9098abb6ed5f152458338aa73e"}
Jul 31 13:14:49.844763 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.844688    2532 scope.go:117] "RemoveContainer" containerID="054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"
Jul 31 13:14:49.854103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.854086    2532 scope.go:117] "RemoveContainer" containerID="054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"
Jul 31 13:14:49.854348 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:14:49.854329    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19\": container with ID starting with 054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19 not found: ID does not exist" containerID="054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"
Jul 31 13:14:49.854407 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.854357    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19"} err="failed to get container status \"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19\": rpc error: code = NotFound desc = could not find container \"054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19\": container with ID starting with 054affc7c99534b01fe61afe0edabdc554a372c5b674ba68a9bad4a0fcd9eb19 not found: ID does not exist"
Jul 31 13:14:49.865870 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.865828    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:14:49.870193 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:49.870161    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fd6ea7-85c4df6f97-6lwwv"]
Jul 31 13:14:51.346139 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:51.346107    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8373fde2-b6ea-47e6-81c8-c0e668dcd67e" path="/var/lib/kubelet/pods/8373fde2-b6ea-47e6-81c8-c0e668dcd67e/volumes"
Jul 31 13:14:51.346548 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:51.346515    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13c777e-e991-4e35-8652-1c22671baf27" path="/var/lib/kubelet/pods/f13c777e-e991-4e35-8652-1c22671baf27/volumes"
Jul 31 13:14:51.750520 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:14:51.750487    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:15:46.211536 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.211497    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:15:46.212073 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.211725    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" podUID="3510203d-81fb-46f4-8af2-c13a99541503" containerName="maas-api" containerID="cri-o://1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb" gracePeriod=30
Jul 31 13:15:46.269342 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.269302    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:15:46.269620 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.269580    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" podUID="916454d0-0479-4a97-8d56-04012cba80b5" containerName="payload-pre-processing" containerID="cri-o://1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029" gracePeriod=30
Jul 31 13:15:46.529050 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.529024    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:15:46.559152 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.559126    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:15:46.563228 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563207    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls\") pod \"3510203d-81fb-46f4-8af2-c13a99541503\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") "
Jul 31 13:15:46.563367 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563265    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca\") pod \"3510203d-81fb-46f4-8af2-c13a99541503\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") "
Jul 31 13:15:46.563367 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563288    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume\") pod \"916454d0-0479-4a97-8d56-04012cba80b5\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") "
Jul 31 13:15:46.563367 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563335    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szfgw\" (UniqueName: \"kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw\") pod \"3510203d-81fb-46f4-8af2-c13a99541503\" (UID: \"3510203d-81fb-46f4-8af2-c13a99541503\") "
Jul 31 13:15:46.563367 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563358    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd562\" (UniqueName: \"kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562\") pod \"916454d0-0479-4a97-8d56-04012cba80b5\" (UID: \"916454d0-0479-4a97-8d56-04012cba80b5\") "
Jul 31 13:15:46.563747 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563720    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3510203d-81fb-46f4-8af2-c13a99541503" (UID: "3510203d-81fb-46f4-8af2-c13a99541503"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:15:46.563903 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.563727    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "916454d0-0479-4a97-8d56-04012cba80b5" (UID: "916454d0-0479-4a97-8d56-04012cba80b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:15:46.565533 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.565502    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw" (OuterVolumeSpecName: "kube-api-access-szfgw") pod "3510203d-81fb-46f4-8af2-c13a99541503" (UID: "3510203d-81fb-46f4-8af2-c13a99541503"). InnerVolumeSpecName "kube-api-access-szfgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:15:46.565767 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.565744    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3510203d-81fb-46f4-8af2-c13a99541503" (UID: "3510203d-81fb-46f4-8af2-c13a99541503"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 13:15:46.565872 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.565766    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562" (OuterVolumeSpecName: "kube-api-access-kd562") pod "916454d0-0479-4a97-8d56-04012cba80b5" (UID: "916454d0-0479-4a97-8d56-04012cba80b5"). InnerVolumeSpecName "kube-api-access-kd562". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:15:46.664318 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.664279    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szfgw\" (UniqueName: \"kubernetes.io/projected/3510203d-81fb-46f4-8af2-c13a99541503-kube-api-access-szfgw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:15:46.664318 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.664314    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd562\" (UniqueName: \"kubernetes.io/projected/916454d0-0479-4a97-8d56-04012cba80b5-kube-api-access-kd562\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:15:46.664318 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.664325    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3510203d-81fb-46f4-8af2-c13a99541503-maas-api-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:15:46.664559 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.664336    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3510203d-81fb-46f4-8af2-c13a99541503-openshift-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:15:46.664559 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:46.664350    2532 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/916454d0-0479-4a97-8d56-04012cba80b5-plugins-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:15:47.182998 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.182959    2532 generic.go:356] "Generic (PLEG): container finished" podID="916454d0-0479-4a97-8d56-04012cba80b5" containerID="1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029" exitCode=0
Jul 31 13:15:47.183229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.183008    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"
Jul 31 13:15:47.183229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.183041    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" event={"ID":"916454d0-0479-4a97-8d56-04012cba80b5","Type":"ContainerDied","Data":"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"}
Jul 31 13:15:47.183229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.183105    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8" event={"ID":"916454d0-0479-4a97-8d56-04012cba80b5","Type":"ContainerDied","Data":"8bb42e6a06e3b6034e8afa7a21d83c24f78bc349d6a6987ed11e0911abd9c630"}
Jul 31 13:15:47.183229 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.183129    2532 scope.go:117] "RemoveContainer" containerID="1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"
Jul 31 13:15:47.184372 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.184354    2532 generic.go:356] "Generic (PLEG): container finished" podID="3510203d-81fb-46f4-8af2-c13a99541503" containerID="1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb" exitCode=0
Jul 31 13:15:47.184461 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.184396    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" event={"ID":"3510203d-81fb-46f4-8af2-c13a99541503","Type":"ContainerDied","Data":"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"}
Jul 31 13:15:47.184461 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.184402    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"
Jul 31 13:15:47.184461 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.184429    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8" event={"ID":"3510203d-81fb-46f4-8af2-c13a99541503","Type":"ContainerDied","Data":"f81934af59144ae54111bb5786c5638e26adc410d061d014d40b7a7dacb51c79"}
Jul 31 13:15:47.193130 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.193110    2532 scope.go:117] "RemoveContainer" containerID="1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"
Jul 31 13:15:47.193479 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:15:47.193455    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029\": container with ID starting with 1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029 not found: ID does not exist" containerID="1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"
Jul 31 13:15:47.193565 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.193484    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029"} err="failed to get container status \"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029\": rpc error: code = NotFound desc = could not find container \"1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029\": container with ID starting with 1d22040da46b64599282db95fc15302ce052ddd625b2715dbc8c557854353029 not found: ID does not exist"
Jul 31 13:15:47.193565 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.193502    2532 scope.go:117] "RemoveContainer" containerID="1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"
Jul 31 13:15:47.202729 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.202709    2532 scope.go:117] "RemoveContainer" containerID="1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"
Jul 31 13:15:47.203030 ip-10-0-141-165 kubenswrapper[2532]: E0731 13:15:47.203008    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb\": container with ID starting with 1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb not found: ID does not exist" containerID="1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"
Jul 31 13:15:47.203114 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.203036    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb"} err="failed to get container status \"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb\": rpc error: code = NotFound desc = could not find container \"1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb\": container with ID starting with 1a7a28b9918113be36c74920b850bde2a222c77ebc59540dfaeac3e6266a3edb not found: ID does not exist"
Jul 31 13:15:47.210658 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.210637    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:15:47.214416 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.214393    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f59489-5c8bf885fb-jd8c8"]
Jul 31 13:15:47.222739 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.222718    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:15:47.228958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.228936    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f59489-567f75c9b6-v84z8"]
Jul 31 13:15:47.346607 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.346572    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3510203d-81fb-46f4-8af2-c13a99541503" path="/var/lib/kubelet/pods/3510203d-81fb-46f4-8af2-c13a99541503/volumes"
Jul 31 13:15:47.346972 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:47.346959    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="916454d0-0479-4a97-8d56-04012cba80b5" path="/var/lib/kubelet/pods/916454d0-0479-4a97-8d56-04012cba80b5/volumes"
Jul 31 13:15:48.348494 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:15:48.348453    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:16:02.456956 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:02.456915    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:16:13.284877 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.284838    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:16:13.285360 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.285119    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" podUID="bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" containerName="payload-processing" containerID="cri-o://d5b7e1dfcb93bc49d1cda38960bf595bb0dd02df986c02427e8c6b7bb36fd891" gracePeriod=30
Jul 31 13:16:13.536076 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.536014    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:16:13.622819 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.622782    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume\") pod \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") "
Jul 31 13:16:13.623001 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.622875    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgzjx\" (UniqueName: \"kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx\") pod \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\" (UID: \"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5\") "
Jul 31 13:16:13.623161 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.623128    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" (UID: "bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 13:16:13.624832 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.624810    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx" (OuterVolumeSpecName: "kube-api-access-fgzjx") pod "bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" (UID: "bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5"). InnerVolumeSpecName "kube-api-access-fgzjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 13:16:13.723740 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.723707    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgzjx\" (UniqueName: \"kubernetes.io/projected/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-kube-api-access-fgzjx\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:16:13.723740 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:13.723735    2532 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5-plugins-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 31 13:16:14.021782 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.021750    2532 scope.go:117] "RemoveContainer" containerID="d5b7e1dfcb93bc49d1cda38960bf595bb0dd02df986c02427e8c6b7bb36fd891"
Jul 31 13:16:14.352506 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.352421    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"
Jul 31 13:16:14.352896 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.352426    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" event={"ID":"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5","Type":"ContainerDied","Data":"d5b7e1dfcb93bc49d1cda38960bf595bb0dd02df986c02427e8c6b7bb36fd891"}
Jul 31 13:16:14.352896 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.352551    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx" event={"ID":"bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5","Type":"ContainerDied","Data":"73791f5849fa7c3058b0c44c2dde9f0af17c037032851499747b322658193e5f"}
Jul 31 13:16:14.385339 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.385309    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:16:14.388023 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:14.388001    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e55b67-6bfdb7d9f8-wv5wx"]
Jul 31 13:16:15.346299 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:15.346269    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" path="/var/lib/kubelet/pods/bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5/volumes"
Jul 31 13:16:15.443931 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:15.443894    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-n4qdf"]
Jul 31 13:16:26.322742 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:26.322709    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mbvsh_cf2c0d9e-cd8e-4612-932b-a3a72be75039/manager/0.log"
Jul 31 13:16:27.055757 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:27.055732    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996b4548f-xt67k_6ae3afb5-942b-497f-8dec-c438bde3f551/copy-manifests/0.log"
Jul 31 13:16:27.061647 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:27.061617    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996b4548f-xt67k_6ae3afb5-942b-497f-8dec-c438bde3f551/manager/0.log"
Jul 31 13:16:29.111044 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:29.111007    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-ptxn7_96b8acde-96f6-430a-8f17-ac8130890767/manager/0.log"
Jul 31 13:16:29.219351 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:29.219321    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-n4qdf_c601995e-a55e-431a-a3c6-6ab45a014143/limitador/0.log"
Jul 31 13:16:31.018944 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:31.018912    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qnxnt_3fccc59d-9a0d-45a6-9138-b71479d27aab/main/0.log"
Jul 31 13:16:31.249060 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:31.249029    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6fvrc_7b73da28-0877-4d16-b8f9-1fb21b361a85/main/0.log"
Jul 31 13:16:31.471810 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:31.471777    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-g85tf_8717eb71-9a2b-4d82-8475-ec0cd0a891c3/main/0.log"
Jul 31 13:16:39.424197 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:39.424104    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xqkmw_01539827-8873-463d-89b1-ff47d4e4b2a8/global-pull-secret-syncer/0.log"
Jul 31 13:16:39.501006 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:39.500969    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j6kr9_a66df52f-4829-48d2-b940-6eba62a720b9/konnectivity-agent/0.log"
Jul 31 13:16:39.606270 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:39.606239    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-165.ec2.internal_e7711a431e63758ca060bb86d740f9a6/haproxy/0.log"
Jul 31 13:16:44.954334 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:44.954296    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-ptxn7_96b8acde-96f6-430a-8f17-ac8130890767/manager/0.log"
Jul 31 13:16:44.973899 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:44.973867    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-n4qdf_c601995e-a55e-431a-a3c6-6ab45a014143/limitador/0.log"
Jul 31 13:16:46.364024 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.363995    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/alertmanager/0.log"
Jul 31 13:16:46.384771 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.384749    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/config-reloader/0.log"
Jul 31 13:16:46.404305 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.404285    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/kube-rbac-proxy-web/0.log"
Jul 31 13:16:46.442077 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.442057    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/kube-rbac-proxy/0.log"
Jul 31 13:16:46.458144 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.458121    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/kube-rbac-proxy-metric/0.log"
Jul 31 13:16:46.483132 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.483111    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/prom-label-proxy/0.log"
Jul 31 13:16:46.504390 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.504371    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54a39662-6818-4787-a665-ebeb60173157/init-config-reloader/0.log"
Jul 31 13:16:46.547398 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.547375    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-q7fd2_e553f719-737d-46b1-b8e7-b58473ee0a60/cluster-monitoring-operator/0.log"
Jul 31 13:16:46.641200 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.641109    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6df59b9d97-mmh8z_76882839-9656-4ba4-9f92-8c26a2157e9c/metrics-server/0.log"
Jul 31 13:16:46.674288 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.674261    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-8dt42_050fd95a-c892-42e3-b840-8d852f94864d/monitoring-plugin/0.log"
Jul 31 13:16:46.734893 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.734870    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nm4_6664d00e-e027-4f7c-a1d2-fa4259507924/node-exporter/0.log"
Jul 31 13:16:46.761385 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.761364    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nm4_6664d00e-e027-4f7c-a1d2-fa4259507924/kube-rbac-proxy/0.log"
Jul 31 13:16:46.782092 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:46.782075    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nm4_6664d00e-e027-4f7c-a1d2-fa4259507924/init-textfile/0.log"
Jul 31 13:16:47.270129 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.270100    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b658575dc-qvnpz_b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a/telemeter-client/0.log"
Jul 31 13:16:47.290201 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.290161    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b658575dc-qvnpz_b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a/reload/0.log"
Jul 31 13:16:47.411078 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.411051    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b658575dc-qvnpz_b74ca37b-1169-44ab-b8dc-7ea21b1c3b1a/kube-rbac-proxy/0.log"
Jul 31 13:16:47.445857 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.445832    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/thanos-query/0.log"
Jul 31 13:16:47.473367 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.473345    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/kube-rbac-proxy-web/0.log"
Jul 31 13:16:47.500050 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.500026    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/kube-rbac-proxy/0.log"
Jul 31 13:16:47.521958 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.521898    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/prom-label-proxy/0.log"
Jul 31 13:16:47.545555 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.545533    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/kube-rbac-proxy-rules/0.log"
Jul 31 13:16:47.566848 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:47.566826    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c49f57fc4-vrj8f_5459e6e5-1a04-483a-ae95-51bfb7d7f23c/kube-rbac-proxy-metrics/0.log"
Jul 31 13:16:49.346506 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346479    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"]
Jul 31 13:16:49.346875 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346839    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8373fde2-b6ea-47e6-81c8-c0e668dcd67e" containerName="revoke-keys"
Jul 31 13:16:49.346875 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346850    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="8373fde2-b6ea-47e6-81c8-c0e668dcd67e" containerName="revoke-keys"
Jul 31 13:16:49.346946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346888    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="916454d0-0479-4a97-8d56-04012cba80b5" containerName="payload-pre-processing"
Jul 31 13:16:49.346946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346896    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="916454d0-0479-4a97-8d56-04012cba80b5" containerName="payload-pre-processing"
Jul 31 13:16:49.346946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346938    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13c777e-e991-4e35-8652-1c22671baf27" containerName="maas-api"
Jul 31 13:16:49.346946 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346945    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13c777e-e991-4e35-8652-1c22671baf27" containerName="maas-api"
Jul 31 13:16:49.347103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346966    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3510203d-81fb-46f4-8af2-c13a99541503" containerName="maas-api"
Jul 31 13:16:49.347103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.346972    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="3510203d-81fb-46f4-8af2-c13a99541503" containerName="maas-api"
Jul 31 13:16:49.347103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347010    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" containerName="payload-processing"
Jul 31 13:16:49.347103 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347016    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" containerName="payload-processing"
Jul 31 13:16:49.347569 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347115    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="8373fde2-b6ea-47e6-81c8-c0e668dcd67e" containerName="revoke-keys"
Jul 31 13:16:49.347569 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347128    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="916454d0-0479-4a97-8d56-04012cba80b5" containerName="payload-pre-processing"
Jul 31 13:16:49.347569 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347140    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13c777e-e991-4e35-8652-1c22671baf27" containerName="maas-api"
Jul 31 13:16:49.347569 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347152    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="3510203d-81fb-46f4-8af2-c13a99541503" containerName="maas-api"
Jul 31 13:16:49.347569 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.347162    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd4a0b0-5adf-4e7d-aed2-548a6b2b65f5" containerName="payload-processing"
Jul 31 13:16:49.350785 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.350764    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"]
Jul 31 13:16:49.350892 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.350852    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.353254 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.353231    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6r5k\"/\"kube-root-ca.crt\""
Jul 31 13:16:49.354031 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.353995    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c6r5k\"/\"default-dockercfg-bpt22\""
Jul 31 13:16:49.354146 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.354042    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6r5k\"/\"openshift-service-ca.crt\""
Jul 31 13:16:49.433076 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.433040    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qh6c\" (UniqueName: \"kubernetes.io/projected/938760c2-e6a9-4492-a166-7e87e3d433da-kube-api-access-6qh6c\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.433218 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.433108    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-lib-modules\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.433218 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.433142    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-sys\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.433331 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.433238    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-proc\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.433331 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.433271    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-podres\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.533847 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.533820    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-proc\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.533970 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.533853    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-podres\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.533970 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.533951    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-proc\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.533980    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qh6c\" (UniqueName: \"kubernetes.io/projected/938760c2-e6a9-4492-a166-7e87e3d433da-kube-api-access-6qh6c\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.534037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-lib-modules\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.534052    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-podres\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534084 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.534064    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-sys\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534306 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.534122    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-lib-modules\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.534306 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.534145    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938760c2-e6a9-4492-a166-7e87e3d433da-sys\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.541706 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.541688    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qh6c\" (UniqueName: \"kubernetes.io/projected/938760c2-e6a9-4492-a166-7e87e3d433da-kube-api-access-6qh6c\") pod \"perf-node-gather-daemonset-hbv6q\" (UID: \"938760c2-e6a9-4492-a166-7e87e3d433da\") " pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.661856 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.661785    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:49.791033 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:49.790986    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"]
Jul 31 13:16:49.792510 ip-10-0-141-165 kubenswrapper[2532]: W0731 13:16:49.792480    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod938760c2_e6a9_4492_a166_7e87e3d433da.slice/crio-a222a80ffb4f5765784198cf1fea80f5defd95447c63d6b06c4ab5ea0d20e183 WatchSource:0}: Error finding container a222a80ffb4f5765784198cf1fea80f5defd95447c63d6b06c4ab5ea0d20e183: Status 404 returned error can't find the container with id a222a80ffb4f5765784198cf1fea80f5defd95447c63d6b06c4ab5ea0d20e183
Jul 31 13:16:50.018933 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.018905    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-597b994885-fs958_75a04fb6-f383-4c40-8a5c-f59e2b455239/console/0.log"
Jul 31 13:16:50.057025 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.057000    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-8m6n2_f6203059-b243-4da7-9f75-a2464d2173e6/download-server/0.log"
Jul 31 13:16:50.549843 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.549804    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q" event={"ID":"938760c2-e6a9-4492-a166-7e87e3d433da","Type":"ContainerStarted","Data":"95c0f49a40b45d39f52973f3804400eed7ff92c1b6e3fa554ff861879e56ff5e"}
Jul 31 13:16:50.550219 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.549859    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q" event={"ID":"938760c2-e6a9-4492-a166-7e87e3d433da","Type":"ContainerStarted","Data":"a222a80ffb4f5765784198cf1fea80f5defd95447c63d6b06c4ab5ea0d20e183"}
Jul 31 13:16:50.550219 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.549884    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:50.565539 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.565493    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q" podStartSLOduration=1.5654772860000001 podStartE2EDuration="1.565477286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 13:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 13:16:50.563996773 +0000 UTC m=+3697.795861528" watchObservedRunningTime="2026-07-31 13:16:50.565477286 +0000 UTC m=+3697.797342091"
Jul 31 13:16:50.755768 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:50.755742    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-8rvn4_d7843221-f183-4ddd-b2d4-e4ded5d90b8d/volume-data-source-validator/0.log"
Jul 31 13:16:51.790822 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:51.790795    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wz22_de449ca7-7464-4981-825f-9c3fc3dead90/dns/0.log"
Jul 31 13:16:51.811109 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:51.811081    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wz22_de449ca7-7464-4981-825f-9c3fc3dead90/kube-rbac-proxy/0.log"
Jul 31 13:16:51.923318 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:51.923292    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9lkp6_81c723fb-b45f-49ee-9a71-0f00ecb73e6b/dns-node-resolver/0.log"
Jul 31 13:16:52.544285 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:52.544254    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dgnbk_50430d0c-78a9-4876-8dfe-9753de79bb9a/node-ca/0.log"
Jul 31 13:16:54.679825 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:54.679788    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zlkhg_1107bc35-a8da-4171-a247-271408b87167/serve-healthcheck-canary/0.log"
Jul 31 13:16:55.143454 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:55.143424    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-vgntt_8bd46abd-03c1-4f7d-88d0-8957be2aeb8e/insights-operator/0.log"
Jul 31 13:16:55.144283 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:55.144266    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-vgntt_8bd46abd-03c1-4f7d-88d0-8957be2aeb8e/insights-operator/1.log"
Jul 31 13:16:55.162779 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:55.162752    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2w2jf_65406aef-7619-4ecb-8cc3-8e5783a0090f/kube-rbac-proxy/0.log"
Jul 31 13:16:55.181183 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:55.181157    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2w2jf_65406aef-7619-4ecb-8cc3-8e5783a0090f/exporter/0.log"
Jul 31 13:16:55.199871 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:55.199848    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2w2jf_65406aef-7619-4ecb-8cc3-8e5783a0090f/extractor/0.log"
Jul 31 13:16:56.567106 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:56.567077    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c6r5k/perf-node-gather-daemonset-hbv6q"
Jul 31 13:16:57.594745 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:57.594708    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mbvsh_cf2c0d9e-cd8e-4612-932b-a3a72be75039/manager/0.log"
Jul 31 13:16:58.591975 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:58.591943    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996b4548f-xt67k_6ae3afb5-942b-497f-8dec-c438bde3f551/manager/0.log"
Jul 31 13:16:58.612725 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:16:58.612702    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996b4548f-xt67k_6ae3afb5-942b-497f-8dec-c438bde3f551/copy-manifests/0.log"
Jul 31 13:17:00.059616 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:00.059589    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f679cb99-gr9lx_c4d0824d-6111-4585-980c-fd9f20ff597f/manager/0.log"
Jul 31 13:17:00.089990 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:00.089967    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4l8gc_f1364e44-b10e-4c4b-b6cd-f8e456e9c2a4/openshift-lws-operator/0.log"
Jul 31 13:17:05.013963 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:05.013901    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-b2945_76a9a985-ece6-482a-9aa7-824fc8c45a48/migrator/0.log"
Jul 31 13:17:05.032462 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:05.032433    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-b2945_76a9a985-ece6-482a-9aa7-824fc8c45a48/graceful-termination/0.log"
Jul 31 13:17:05.367749 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:05.367671    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-sbf7v_8562adee-da47-4daf-8786-144e50002130/kube-storage-version-migrator-operator/1.log"
Jul 31 13:17:05.368533 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:05.368514    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-sbf7v_8562adee-da47-4daf-8786-144e50002130/kube-storage-version-migrator-operator/0.log"
Jul 31 13:17:06.610364 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.610335    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/kube-multus-additional-cni-plugins/0.log"
Jul 31 13:17:06.628474 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.628448    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/egress-router-binary-copy/0.log"
Jul 31 13:17:06.647417 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.647391    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/cni-plugins/0.log"
Jul 31 13:17:06.665778 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.665758    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/bond-cni-plugin/0.log"
Jul 31 13:17:06.684140 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.684117    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/routeoverride-cni/0.log"
Jul 31 13:17:06.702935 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.702915    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/whereabouts-cni-bincopy/0.log"
Jul 31 13:17:06.721066 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.721044    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t775z_49bfc914-42f7-4768-9448-20c7ce0121ce/whereabouts-cni/0.log"
Jul 31 13:17:06.809905 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.809884    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zwgfj_fd1c837c-3d3d-43f7-a61d-8a8774cd91cb/kube-multus/0.log"
Jul 31 13:17:06.829094 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.829068    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-59h74_f581f940-84e7-41b3-a723-29e3e33fceac/network-metrics-daemon/0.log"
Jul 31 13:17:06.847316 ip-10-0-141-165 kubenswrapper[2532]: I0731 13:17:06.847299    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-59h74_f581f940-84e7-41b3-a723-29e3e33fceac/kube-rbac-proxy/0.log"