Jul 06 01:18:10.279370 ip-10-0-133-137 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 01:18:10.279378 ip-10-0-133-137 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 01:18:10.279386 ip-10-0-133-137 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 01:18:10.279593 ip-10-0-133-137 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 01:18:20.287979 ip-10-0-133-137 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 01:18:20.287998 ip-10-0-133-137 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f8c87c76479d41e7aba1692e91fa8969 --
Jul 06 01:19:59.012507 ip-10-0-133-137 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 01:19:59.446503 ip-10-0-133-137 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 01:19:59.446503 ip-10-0-133-137 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 01:19:59.446503 ip-10-0-133-137 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 01:19:59.446503 ip-10-0-133-137 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 01:19:59.446503 ip-10-0-133-137 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 01:19:59.447196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.446561    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 01:19:59.451131 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451116    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.451131 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451131    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451135    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451139    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451142    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451154    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451159    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451162    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451164    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451167    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451170    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451173    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451175    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451178    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451180    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451185    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451188    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451191    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451194    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451197    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.451193 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451200    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451203    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451206    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451209    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451224    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451227    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451230    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451233    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451235    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451239    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451242    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451245    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451248    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451250    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451253    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451255    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451258    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451260    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451263    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451266    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.451668 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451268    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451270    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451273    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451277    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451280    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451284    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451287    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451289    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451292    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451294    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451296    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451299    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451302    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451304    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451308    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451310    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451313    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451316    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451318    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.452146 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451321    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451323    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451326    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451328    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451331    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451333    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451336    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451338    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451341    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451343    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451345    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451348    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451351    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451353    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451356    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451359    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451362    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451365    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451368    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451370    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.452627 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451373    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451375    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451378    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451380    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451383    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451385    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451388    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451390    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451393    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451395    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451398    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451400    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451403    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451406    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451408    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451411    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451413    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451415    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451418    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.451420    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.453170 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452087    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452093    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452096    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452099    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452104    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452108    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452111    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452115    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452118    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452121    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452124    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452126    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452129    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452131    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452134    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452137    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452141    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452143    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452146    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.453679 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452149    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452152    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452154    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452157    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452160    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452162    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452164    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452167    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452169    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452171    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452174    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452176    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452179    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452181    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452183    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452186    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452188    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452190    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452192    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452196    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.454145 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452198    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452201    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452203    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452205    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452208    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452227    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452230    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452233    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452236    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452239    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452242    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452245    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452248    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452251    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452254    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452257    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452259    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452262    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452265    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452267    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.454640 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452270    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452272    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452274    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452277    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452279    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452281    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452284    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452286    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452289    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452291    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452293    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452296    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452299    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452302    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452305    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452308    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452310    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452313    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452315    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452318    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.455138 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452320    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452323    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452325    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452328    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452331    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452333    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452336    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452338    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452340    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452343    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452346    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452348    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452351    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452353    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452356    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452358    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452360    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452362    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452365    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452367    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.455647 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452769    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452775    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452778    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452783    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452787    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452790    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452792    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452795    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452797    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452800    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452802    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452805    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452807    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452810    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452812    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452815    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452817    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452820    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452822    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452825    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.456124 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452827    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452829    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452832    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452834    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452837    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452839    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452841    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452843    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452846    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452848    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452851    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452854    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452858    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452861    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452877    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452882    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452884    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452887    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452890    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452892    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.456705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452895    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452897    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452899    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452902    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452904    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452907    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452909    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452912    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452915    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452917    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452920    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452923    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452926    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452928    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452931    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452933    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452936    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452938    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452940    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452943    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.457208 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452945    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452948    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452950    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452952    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452955    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452957    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452961    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452966    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452968    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452971    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452974    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452976    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452979    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452981    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452984    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452986    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452990    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452992    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452995    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.452998    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.457700 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453000    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453003    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453005    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453008    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453010    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453013    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453016    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453020    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453022    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453025    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453028    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453031    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453034    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453037    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453039    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453042    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453045    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453047    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453050    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453109    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 01:19:59.458209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453116    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453121    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453125    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453130    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453134    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453137    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453141    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453144    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453147    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453150    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453155    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453158    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453160    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453163    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453166    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453169    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453172    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453176    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453179    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453182    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453185    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453188    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453191    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 01:19:59.458725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453194    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453197    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453201    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453204    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453206    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453209    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453227    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453230    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453237    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453240    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453243    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453246    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453248    2567 flags.go:64] FLAG: --enable-server="true"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453252    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453256    2567 flags.go:64] FLAG: --event-burst="100"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453258    2567 flags.go:64] FLAG: --event-qps="50"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453261    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453264    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453267    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453271    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453274    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453277    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453279    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453282    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453285    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 01:19:59.459304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453287    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453290    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453293    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453296    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453298    2567 flags.go:64] FLAG: --feature-gates=""
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453302    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453305    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453308    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453310    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453314    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453316    2567 flags.go:64] FLAG: --help="false"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453319    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453322    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453325    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453327    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453333    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453336    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453339    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453342    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453344    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453347    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453350    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453353    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453356    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 06 01:19:59.459917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453358    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453361    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453364    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453367    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453369    2567 flags.go:64] FLAG: --lock-file=""
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453372    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453374    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453377    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453382    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453385    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453388    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453390    2567 flags.go:64] FLAG: --logging-format="text"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453393    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453396    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453399    2567 flags.go:64] FLAG: --manifest-url=""
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453401    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453405    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453408    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453412    2567 flags.go:64] FLAG: --max-pods="110"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453415    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453418    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453420    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453424    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453432    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453436    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 01:19:59.460589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453439    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453446    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453449    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453452    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453455    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453458    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453463    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453466    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453469    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453472    2567 flags.go:64] FLAG: --port="10250"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453475    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453478    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06967d6c9f5a06534"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453481    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453484    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453487    2567 flags.go:64] FLAG: --register-node="true"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453490    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453494    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453496    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453499    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453501    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453505    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453508    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453511    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453513    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453516    2567 flags.go:64] FLAG: --runonce="false"
Jul 06 01:19:59.461194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453519    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453522    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453524    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453527    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453529    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453532    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453536    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453539    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453541    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453544    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453546    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453549    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453552    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453555    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453557    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453562    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453565    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453568    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453572    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453574    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453577    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453580    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453582    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453585    2567 flags.go:64] FLAG: --v="2"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453589    2567 flags.go:64] FLAG: --version="false"
Jul 06 01:19:59.461848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453593    2567 flags.go:64] FLAG: --vmodule=""
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453596    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.453599    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453703    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453708    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453712    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453715    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453717    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453720    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453722    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453725    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453727    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453731    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453735    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453738    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453741    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453744    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453746    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453749    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.462469 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453751    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453753    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453756    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453759    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453763    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453765    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453768    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453770    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453772    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453775    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453778    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453781    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453784    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453788    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453790    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453793    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453796    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453799    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453801    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.462931 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453804    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453806    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453809    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453811    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453813    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453816    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453818    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453821    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453823    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453826    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453828    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453831    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453833    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453836    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453838    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453840    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453843    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453847    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453849    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453851    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.463427 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453854    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453856    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453858    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453861    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453863    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453866    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453868    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453870    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453873    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453875    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453878    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453880    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453883    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453885    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453887    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453890    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453892    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453895    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453897    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453899    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.463935 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453902    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453904    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453906    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453909    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453911    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453914    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453916    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453919    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453921    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453926    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453928    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453930    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453933    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453935    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453937    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453940    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453942    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453945    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453947    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453949    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.464503 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453952    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453954    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453956    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.453959    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.455027    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.461938    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.461952    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.461970    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.461976    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462030    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462034    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462038    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462041    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462043    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462046    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.464990 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462049    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462051    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462054    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462057    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462060    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462062    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462065    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462068    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462070    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462074    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462076    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462078    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462081    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462083    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462086    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462089    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462091    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462094    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462096    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462098    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.465388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462102    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462106    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462109    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462113    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462116    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462119    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462123    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462126    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462129    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462131    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462134    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462137    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462139    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462142    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462145    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462149    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462153    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462155    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462157    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462160    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.465880 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462163    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462165    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462167    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462170    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462172    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462174    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462177    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462179    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462182    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462186    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462189    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462192    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462195    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462197    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462200    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462203    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462206    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462208    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462225    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.466382 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462228    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462230    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462233    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462236    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462238    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462240    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462243    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462245    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462248    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462250    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462253    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462255    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462257    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462260    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462263    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462265    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462268    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462270    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462273    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.466857 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462275    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462278    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462280    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462283    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462285    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462287    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462290    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462292    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462295    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462298    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462300    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462303    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462305    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462308    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462311    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.467388 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.462316    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462421    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462425    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462427    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462430    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462433    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462435    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462438    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462440    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462443    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462446    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462450    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462453    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462456    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462459    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462461    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462463    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462467    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 01:19:59.467775 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462470    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462473    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462477    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462480    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462482    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462485    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462487    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462490    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462492    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462495    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462497    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462500    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462502    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462504    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462507    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462509    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462512    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462514    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462517    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462519    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 01:19:59.468204 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462521    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462524    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462526    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462528    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462531    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462533    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462535    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462538    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462541    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462543    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462546    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462548    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462551    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462554    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462556    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462559    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462561    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462563    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462566    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462568    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 01:19:59.468743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462571    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462589    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462592    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462595    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462598    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462601    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462603    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462605    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462608    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462610    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462613    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462615    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462617    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462620    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462622    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462624    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462627    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462629    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462632    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462634    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 01:19:59.469221 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462640    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462642    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462645    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462647    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462650    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462652    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462655    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462657    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462659    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462662    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462664    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462667    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462669    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462672    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462674    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462676    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462680    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462683    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462685    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 01:19:59.469712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462688    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462690    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:19:59.462693    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.462697    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.462787    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.465080    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.466645    2567 server.go:1019] "Starting client certificate rotation"
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.466749    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.466778    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 01:19:59.470163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.469685    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 01:19:59.476238 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.476194    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 01:19:59.489413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.489397    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 06 01:19:59.492621 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.492608    2567 log.go:25] "Validated CRI v1 image API"
Jul 06 01:19:59.493452 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.493439    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 01:19:59.496103 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.496084    2567 fs.go:135] Filesystem UUIDs: map[05500fe3-a6a9-405c-b2a3-8bb4594d5f58:/dev/nvme0n1p4 795966e3-3e4f-427a-926e-75e0bdd10cdd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 06 01:19:59.496176 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.496104    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 01:19:59.500312 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.500184    2567 manager.go:216] Machine: {Timestamp:2026-07-06 01:19:59.499723091 +0000 UTC m=+0.356778868 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097447 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b4bbd1acb85f9f16df0fd176839d SystemUUID:ec26b4bb-d1ac-b85f-9f16-df0fd176839d BootID:f8c87c76-479d-41e7-aba1-692e91fa8969 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f0:00:d3:e5:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f0:00:d3:e5:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:1c:f7:d1:b6:03 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 01:19:59.500312 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.500304    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 01:19:59.500464 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.500393    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 01:19:59.502439 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.502424    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 01:19:59.503730 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.503710    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 01:19:59.503856 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.503733    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 01:19:59.504562 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.504550    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 01:19:59.504562 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.504563    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 01:19:59.504632 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.504571    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 01:19:59.504632 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.504580    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 01:19:59.506329 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.506319    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 01:19:59.509751 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509741    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 01:19:59.509853 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509844    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 01:19:59.509882 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509872    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 01:19:59.509917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509882    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 01:19:59.509917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509901    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 01:19:59.509917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509909    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 06 01:19:59.509917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.509918    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 01:19:59.510207 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.510197    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 01:19:59.510266 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.510229    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 01:19:59.512638 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.512624    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 01:19:59.514976 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.514959    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 01:19:59.515027 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.514994    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 01:19:59.516397 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516381    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 01:19:59.516441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516411    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 01:19:59.516441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516419    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 01:19:59.516441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516425    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 01:19:59.516441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516434    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 01:19:59.516441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516440    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516448    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516453    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516460    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516493    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516503    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 01:19:59.516571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.516508    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 01:19:59.517462 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.517451    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 01:19:59.517505 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.517464    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 01:19:59.520681 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.520666    2567 server.go:1268] "Started kubelet"
Jul 06 01:19:59.520776 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.520745    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 01:19:59.521267 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.521105    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 01:19:59.521445 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.521431    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 01:19:59.521915 ip-10-0-133-137 systemd[1]: Started Kubernetes Kubelet.
Jul 06 01:19:59.526206 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.526185    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 01:19:59.527508 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.527342    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 01:19:59.528295 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.528230    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 01:19:59.533874 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.533853    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 01:19:59.533965 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.533949    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-137.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 01:19:59.534021 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.533980    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 01:19:59.534816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.534796    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 01:19:59.534816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.534804    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 01:19:59.534948 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.534911    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.535492 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.534043    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-137.ec2.internal.18bf8ea0fb17e2a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-137.ec2.internal,UID:ip-10-0-133-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-137.ec2.internal,},FirstTimestamp:2026-07-06 01:19:59.520563881 +0000 UTC m=+0.377619647,LastTimestamp:2026-07-06 01:19:59.520563881 +0000 UTC m=+0.377619647,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-137.ec2.internal,}"
Jul 06 01:19:59.535613 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535597    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 01:19:59.535681 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535633    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 01:19:59.535681 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535649    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 01:19:59.535934 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535752    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 01:19:59.535934 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535762    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 01:19:59.536015 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535978    2567 factory.go:153] Registering CRI-O factory
Jul 06 01:19:59.536015 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.535991    2567 factory.go:223] Registration of the crio container factory successfully
Jul 06 01:19:59.536107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536045    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 01:19:59.536107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536056    2567 factory.go:55] Registering systemd factory
Jul 06 01:19:59.536107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536063    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 06 01:19:59.536107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536081    2567 factory.go:103] Registering Raw factory
Jul 06 01:19:59.536107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536093    2567 manager.go:1196] Started watching for new ooms in manager
Jul 06 01:19:59.536724 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.536695    2567 manager.go:349] Starting recovery of all containers
Jul 06 01:19:59.537351 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.537311    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 01:19:59.537622 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.537596    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 01:19:59.545513 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.545409    2567 manager.go:354] Recovery completed
Jul 06 01:19:59.549571 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.549554    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.551538 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.551514    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.551624 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.551560    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.551624 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.551571    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.552081 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552065    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 01:19:59.552081 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552076    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.552081 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552080    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 01:19:59.552256 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552099    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 01:19:59.552345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552329    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzgq5"
Jul 06 01:19:59.552864 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552850    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.552923 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552882    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.552923 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.552898    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.553496 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.553484    2567 policy_none.go:49] "None policy: Start"
Jul 06 01:19:59.553544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.553500    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 01:19:59.553544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.553511    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 01:19:59.559111 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.559095    2567 policy_none.go:47] "Start"
Jul 06 01:19:59.560490 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.560474    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzgq5"
Jul 06 01:19:59.562787 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.562722    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-137.ec2.internal.18bf8ea0fcf078e4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-137.ec2.internal,UID:ip-10-0-133-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-137.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-137.ec2.internal,},FirstTimestamp:2026-07-06 01:19:59.551535332 +0000 UTC m=+0.408591101,LastTimestamp:2026-07-06 01:19:59.551535332 +0000 UTC m=+0.408591101,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-137.ec2.internal,}"
Jul 06 01:19:59.596759 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.596743    2567 manager.go:337] "Starting Device Plugin manager"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.596775    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.596785    2567 server.go:87] "Starting device plugin registration server"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.597026    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.597035    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.597152    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.597242    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.597252    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.598634    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 01:19:59.618711 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.598680    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.636762 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.636735    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 01:19:59.638048 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.638030    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 01:19:59.638124 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.638066    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 01:19:59.638124 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.638085    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 01:19:59.638124 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.638114    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 01:19:59.640637 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.640621    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 01:19:59.697854 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.697787    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.699198 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.699184    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.699290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.699231    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.699290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.699243    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.699290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.699264    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.708104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.708086    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.708168 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.708106    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-137.ec2.internal\": node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.734038 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.734017    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.738292 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.738276    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"]
Jul 06 01:19:59.738366 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.738341    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.740205 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.740191    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.740303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.740240    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.740303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.740257    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.741618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.741603    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.741728 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.741715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.741780 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.741746    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.742806 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742785    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.742888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742810    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.742888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742821    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.742888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742858    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.742888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742882    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.743068 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.742895    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.744048 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.744033    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.744091 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.744062    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:19:59.744763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.744746    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:19:59.744848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.744777    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:19:59.744848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.744790    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:19:59.768904 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.768884    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-137.ec2.internal\" not found" node="ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.773262 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.773243    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-137.ec2.internal\" not found" node="ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.834125 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.834105    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.837400 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.837383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.837453 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.837407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.837453 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.837423    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c581f67582f2b7493459f83b250c1488-config\") pod \"kube-apiserver-proxy-ip-10-0-133-137.ec2.internal\" (UID: \"c581f67582f2b7493459f83b250c1488\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.934572 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:19:59.934534    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:19:59.937829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c581f67582f2b7493459f83b250c1488-config\") pod \"kube-apiserver-proxy-ip-10-0-133-137.ec2.internal\" (UID: \"c581f67582f2b7493459f83b250c1488\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.937888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.937888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.937950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937906    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.937950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937911    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c581f67582f2b7493459f83b250c1488-config\") pod \"kube-apiserver-proxy-ip-10-0-133-137.ec2.internal\" (UID: \"c581f67582f2b7493459f83b250c1488\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:19:59.938019 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:19:59.937945    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/176a852dc8aa3e063105495696527f75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal\" (UID: \"176a852dc8aa3e063105495696527f75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:20:00.035317 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.035248    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.072628 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.072608    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:20:00.076805 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.076632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:20:00.135374 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.135348    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.235748 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.235711    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.336153 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.336075    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.436587 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.436564    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.465877 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.465860    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 01:20:00.466424 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.465995    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 01:20:00.488889 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.488868    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 01:20:00.521155 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.521134    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 01:20:00.537026 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.537006    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.562770 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.562730    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 01:15:00 +0000 UTC" deadline="2027-12-15 21:36:23.615423976 +0000 UTC"
Jul 06 01:20:00.562770 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.562763    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12668h16m23.052663735s"
Jul 06 01:20:00.577899 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.577879    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 01:20:00.578871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.578854    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 01:20:00.578939 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.578887    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 01:20:00.578939 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.578897    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeHasSufficientPID"
Jul 06 01:20:00.637577 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.637550    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.697540 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:00.697514    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc581f67582f2b7493459f83b250c1488.slice/crio-73bd57c76dbb634dc90bfc25198106af947d379f05c864868413b2cc83dfda95 WatchSource:0}: Error finding container 73bd57c76dbb634dc90bfc25198106af947d379f05c864868413b2cc83dfda95: Status 404 returned error can't find the container with id 73bd57c76dbb634dc90bfc25198106af947d379f05c864868413b2cc83dfda95
Jul 06 01:20:00.702663 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.702649    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 01:20:00.738331 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.738309    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.818743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:00.818724    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176a852dc8aa3e063105495696527f75.slice/crio-86371f87303f1f88f7bc2dd62871d15f16fff0baa58f009c7a60c4126ef44168 WatchSource:0}: Error finding container 86371f87303f1f88f7bc2dd62871d15f16fff0baa58f009c7a60c4126ef44168: Status 404 returned error can't find the container with id 86371f87303f1f88f7bc2dd62871d15f16fff0baa58f009c7a60c4126ef44168
Jul 06 01:20:00.839259 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:00.839238    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-137.ec2.internal\" not found"
Jul 06 01:20:00.885815 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.885777    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 01:20:00.935590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.935568    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal"
Jul 06 01:20:00.947705 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.947692    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 01:20:00.948668 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.948656    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal"
Jul 06 01:20:00.956314 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:00.956291    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 01:20:02.523299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.523267    2567 apiserver.go:52] "Watching apiserver"
Jul 06 01:20:02.531460 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.531428    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 01:20:02.531853 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.531827    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gc54p","kube-system/konnectivity-agent-htk6z","openshift-cluster-node-tuning-operator/tuned-b8zdg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal","openshift-multus/multus-tcksn","openshift-multus/network-metrics-daemon-ktvvs","openshift-network-diagnostics/network-check-target-xlnzs","kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv","openshift-dns/node-resolver-s6r8x","openshift-image-registry/node-ca-26d9w","openshift-multus/multus-additional-cni-plugins-5smz7","openshift-network-operator/iptables-alerter-r7bbj","openshift-ovn-kubernetes/ovnkube-node-tgsgd"]
Jul 06 01:20:02.533530 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.533509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.534734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.534714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.536465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.535711    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 01:20:02.536465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.535886    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.536465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.536103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.536465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.536162    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.536465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.536234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2nxdl\""
Jul 06 01:20:02.537143 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.537110    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 01:20:02.537232 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.537178    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wn6hq\""
Jul 06 01:20:02.537760 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.537327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.538169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.538147    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 01:20:02.538701 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.538684    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.540286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.539880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.541423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.541407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:02.541534 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.541482    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:02.543856 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.543839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.543965 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.543917    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:02.544813 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.544765    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.544916 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.544830    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.545169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545141    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.545281 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545263    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 01:20:02.545634 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545615    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 01:20:02.545761 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545718    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dkpt6\""
Jul 06 01:20:02.545843 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545795    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.545917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545905    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 01:20:02.545990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545718    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vj8qt\""
Jul 06 01:20:02.545990 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.545980    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 01:20:02.546105 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546055    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.546243 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546231    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 01:20:02.546377 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546365    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.546539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546524    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 01:20:02.546698 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546680    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.546895 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546873    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.546979 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.546916    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.546979 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.546965    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:02.547100 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.547082    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n9zdk\""
Jul 06 01:20:02.547364 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.547349    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.547494 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.547480    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 01:20:02.547574 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.547559    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.547744 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.547725    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 01:20:02.548838 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.548822    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.550250 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.550232    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.556286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-registration-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-sys-fs\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-host\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-log-socket\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-system-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.556413 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-cni-binary-copy\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556456    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-netns\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-multus\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-kubelet\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f29e00-0b09-48e9-8218-e1885f613b85-agent-certs\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-tuned\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-etc-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-ovn\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.556739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-os-release\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-bin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-device-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0122480-e48e-449b-b4eb-2f78298552b4-tmp-dir\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556789    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9c475\""
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-run\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-tmp\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-conf-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-systemd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-kubelet-config\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2lw\" (UniqueName: \"kubernetes.io/projected/7bc9f526-fc74-4399-95db-9cd060968c31-kube-api-access-cs2lw\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-netns\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-binary-copy\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.556982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-modprobe-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-conf\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.557328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-var-lib-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6zz5\" (UniqueName: \"kubernetes.io/projected/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kube-api-access-c6zz5\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557059    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-slash\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-bin\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-env-overrides\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ncgj\" (UniqueName: \"kubernetes.io/projected/8373e415-2493-428c-b644-6d236f61f4ee-kube-api-access-5ncgj\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-dbus\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557220    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f29e00-0b09-48e9-8218-e1885f613b85-konnectivity-ca\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-netd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-script-lib\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phs6j\" (UniqueName: \"kubernetes.io/projected/1d794170-f755-4376-842b-17583c87c9c5-kube-api-access-phs6j\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysconfig\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-systemd\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cw5t\" (UniqueName: \"kubernetes.io/projected/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-kube-api-access-7cw5t\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557464    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-cnibin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-etc-kubernetes\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfk9h\" (UniqueName: \"kubernetes.io/projected/e0122480-e48e-449b-b4eb-2f78298552b4-kube-api-access-tfk9h\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgd8b\" (UniqueName: \"kubernetes.io/projected/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-kube-api-access-qgd8b\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60d676d2-25be-4f71-b759-f3e95c48d690-ovn-node-metrics-cert\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-k8s-cni-cncf-io\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-hostroot\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-multus-certs\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d794170-f755-4376-842b-17583c87c9c5-serviceca\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cnibin\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-os-release\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zrmp\" (UniqueName: \"kubernetes.io/projected/60d676d2-25be-4f71-b759-f3e95c48d690-kube-api-access-7zrmp\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557900    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d794170-f755-4376-842b-17583c87c9c5-host\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557924    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.558829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.557993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-multus-daemon-config\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-node-log\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558049    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0122480-e48e-449b-b4eb-2f78298552b4-hosts-file\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-lib-modules\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-kubelet\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-systemd-units\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-config\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-system-cni-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-var-lib-kubelet\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-socket-dir-parent\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-socket-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558470    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-kubernetes\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.559575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.558524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-sys\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.563980 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.563956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 01:20:02.564114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.563956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 01:20:02.564114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.564041    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9mpdd\""
Jul 06 01:20:02.564270 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.564170    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 01:20:02.564270 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.564256    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.564402 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.564386    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 01:20:02.569211 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.569195    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 01:20:02.569329 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.569239    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lgmlw\""
Jul 06 01:20:02.569761 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.569742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4kzbm\""
Jul 06 01:20:02.580989 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.580971    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 01:20:02.605939 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.605919    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k24pk"
Jul 06 01:20:02.613733 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.613709    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k24pk"
Jul 06 01:20:02.648949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.648921    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 01:20:02.655262 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.655216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal" event={"ID":"176a852dc8aa3e063105495696527f75","Type":"ContainerStarted","Data":"86371f87303f1f88f7bc2dd62871d15f16fff0baa58f009c7a60c4126ef44168"}
Jul 06 01:20:02.656662 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.656638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal" event={"ID":"c581f67582f2b7493459f83b250c1488","Type":"ContainerStarted","Data":"73bd57c76dbb634dc90bfc25198106af947d379f05c864868413b2cc83dfda95"}
Jul 06 01:20:02.659541 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-dbus\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.659633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f29e00-0b09-48e9-8218-e1885f613b85-konnectivity-ca\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.659633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-netd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.659633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-script-lib\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.659633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phs6j\" (UniqueName: \"kubernetes.io/projected/1d794170-f755-4376-842b-17583c87c9c5-kube-api-access-phs6j\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysconfig\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-systemd\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cw5t\" (UniqueName: \"kubernetes.io/projected/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-kube-api-access-7cw5t\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659745    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-cnibin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.659783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659772    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-etc-kubernetes\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659823    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfk9h\" (UniqueName: \"kubernetes.io/projected/e0122480-e48e-449b-b4eb-2f78298552b4-kube-api-access-tfk9h\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgd8b\" (UniqueName: \"kubernetes.io/projected/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-kube-api-access-qgd8b\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60d676d2-25be-4f71-b759-f3e95c48d690-ovn-node-metrics-cert\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-k8s-cni-cncf-io\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-hostroot\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.659987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-multus-certs\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660010    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d794170-f755-4376-842b-17583c87c9c5-serviceca\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.660066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cnibin\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-os-release\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660144    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zrmp\" (UniqueName: \"kubernetes.io/projected/60d676d2-25be-4f71-b759-f3e95c48d690-kube-api-access-7zrmp\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d794170-f755-4376-842b-17583c87c9c5-host\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-multus-daemon-config\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-node-log\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0122480-e48e-449b-b4eb-2f78298552b4-hosts-file\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660329    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-lib-modules\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-kubelet\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-systemd-units\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660448    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-config\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l6z9\" (UniqueName: \"kubernetes.io/projected/2d951b3a-08cd-4a50-9c10-e96df76b946e-kube-api-access-2l6z9\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-system-cni-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.660551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-var-lib-kubelet\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-socket-dir-parent\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-socket-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660610    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-kubernetes\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-sys\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-registration-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-sys-fs\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660781    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-host\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-log-socket\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-system-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-cni-binary-copy\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-netns\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.660978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-multus\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661003    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-kubelet\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d951b3a-08cd-4a50-9c10-e96df76b946e-host-slash\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f29e00-0b09-48e9-8218-e1885f613b85-agent-certs\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-tuned\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-etc-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-ovn\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661175    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-os-release\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-bin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d951b3a-08cd-4a50-9c10-e96df76b946e-iptables-alerter-script\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661286    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-device-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661313    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0122480-e48e-449b-b4eb-2f78298552b4-tmp-dir\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-run\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-tmp\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-conf-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-systemd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-kubelet-config\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.661884 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2lw\" (UniqueName: \"kubernetes.io/projected/7bc9f526-fc74-4399-95db-9cd060968c31-kube-api-access-cs2lw\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-netns\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-binary-copy\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-modprobe-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-conf\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-var-lib-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6zz5\" (UniqueName: \"kubernetes.io/projected/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kube-api-access-c6zz5\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-slash\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-bin\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-env-overrides\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.661866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ncgj\" (UniqueName: \"kubernetes.io/projected/8373e415-2493-428c-b644-6d236f61f4ee-kube-api-access-5ncgj\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.662572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.662406    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-dbus\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.662888    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f29e00-0b09-48e9-8218-e1885f613b85-konnectivity-ca\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.662957    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-kubernetes\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.662986    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663005    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-sys\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663043    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-registration-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663049    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-netd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.663151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663080    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-sys-fs\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.663476 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663117    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-log-socket\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.663476 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663205    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-system-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.663652 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663632    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.663716 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663666    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-script-lib\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.663716 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663684    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.663816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663733    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-host\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.663816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.663794    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-cni-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664093 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664075    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysconfig\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.664173 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664153    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-systemd\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.664234 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664160    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-cni-binary-copy\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664287 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664237    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-netns\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664338 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664286    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-multus\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664390 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664340    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-kubelet\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664439 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664407    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-cnibin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664487 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664457    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-etc-kubernetes\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.664753 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.664724    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 01:20:02.665105 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665084    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665524    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-systemd\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665586    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-etc-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665631    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-run-ovn\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665693    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-os-release\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665735    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-var-lib-cni-bin\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665805    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-device-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665849    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-node-log\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665908    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-k8s-cni-cncf-io\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665954    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-hostroot\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.665998    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-host-run-multus-certs\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666063    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0122480-e48e-449b-b4eb-2f78298552b4-tmp-dir\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.666226    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.666295    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:03.166275112 +0000 UTC m=+3.011424504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666455    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d794170-f755-4376-842b-17583c87c9c5-serviceca\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666484    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-run\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666524    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666569    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cnibin\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.668149 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666628    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-os-release\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666847    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-conf-dir\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666878    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d794170-f755-4376-842b-17583c87c9c5-host\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.666937    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667082    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667249    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-ovnkube-config\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667314    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-kubelet\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667366    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-systemd-units\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667412    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-system-cni-dir\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667464    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-var-lib-kubelet\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667592    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0122480-e48e-449b-b4eb-2f78298552b4-hosts-file\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667656    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8373e415-2493-428c-b644-6d236f61f4ee-multus-daemon-config\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667699    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-lib-modules\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667801    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667931    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-socket-dir\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667983    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8373e415-2493-428c-b644-6d236f61f4ee-multus-socket-dir-parent\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.667990    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-run-netns\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668057    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da0c16fd-77bd-48c0-ad86-0e2ae810376f-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.669000 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.668101    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668143    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-modprobe-d\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.668186    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:03.168155344 +0000 UTC m=+3.013304737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668196    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c277b2c-503f-48a2-9429-6ad2c2094008-kubelet-config\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668389    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-sysctl-conf\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668401    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-var-lib-openvswitch\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668480    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-cni-bin\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668617    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-etc-tuned\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668691    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60d676d2-25be-4f71-b759-f3e95c48d690-host-slash\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668826    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60d676d2-25be-4f71-b759-f3e95c48d690-env-overrides\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668933    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-cni-binary-copy\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.668964    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-tmp\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.669619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.669489    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f29e00-0b09-48e9-8218-e1885f613b85-agent-certs\") pod \"konnectivity-agent-htk6z\" (UID: \"53f29e00-0b09-48e9-8218-e1885f613b85\") " pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.670356 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.670331    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60d676d2-25be-4f71-b759-f3e95c48d690-ovn-node-metrics-cert\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.671963 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.671366    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:02.671963 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.671386    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:02.671963 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.671404    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:02.671963 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:02.671472    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:03.171447412 +0000 UTC m=+3.016596820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.679754    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgd8b\" (UniqueName: \"kubernetes.io/projected/f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8-kube-api-access-qgd8b\") pod \"multus-additional-cni-plugins-5smz7\" (UID: \"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8\") " pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.680216    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ncgj\" (UniqueName: \"kubernetes.io/projected/8373e415-2493-428c-b644-6d236f61f4ee-kube-api-access-5ncgj\") pod \"multus-tcksn\" (UID: \"8373e415-2493-428c-b644-6d236f61f4ee\") " pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.680626    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zrmp\" (UniqueName: \"kubernetes.io/projected/60d676d2-25be-4f71-b759-f3e95c48d690-kube-api-access-7zrmp\") pod \"ovnkube-node-tgsgd\" (UID: \"60d676d2-25be-4f71-b759-f3e95c48d690\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.681024    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phs6j\" (UniqueName: \"kubernetes.io/projected/1d794170-f755-4376-842b-17583c87c9c5-kube-api-access-phs6j\") pod \"node-ca-26d9w\" (UID: \"1d794170-f755-4376-842b-17583c87c9c5\") " pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.681470    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2lw\" (UniqueName: \"kubernetes.io/projected/7bc9f526-fc74-4399-95db-9cd060968c31-kube-api-access-cs2lw\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.681851    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfk9h\" (UniqueName: \"kubernetes.io/projected/e0122480-e48e-449b-b4eb-2f78298552b4-kube-api-access-tfk9h\") pod \"node-resolver-s6r8x\" (UID: \"e0122480-e48e-449b-b4eb-2f78298552b4\") " pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.683902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.683844    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cw5t\" (UniqueName: \"kubernetes.io/projected/9abdace0-a781-4cae-8d2b-62bb2ea4c4b0-kube-api-access-7cw5t\") pod \"tuned-b8zdg\" (UID: \"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0\") " pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.686405 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.686362    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6zz5\" (UniqueName: \"kubernetes.io/projected/da0c16fd-77bd-48c0-ad86-0e2ae810376f-kube-api-access-c6zz5\") pod \"aws-ebs-csi-driver-node-j4cfv\" (UID: \"da0c16fd-77bd-48c0-ad86-0e2ae810376f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.762305 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.762276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d951b3a-08cd-4a50-9c10-e96df76b946e-host-slash\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.762466 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.762319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d951b3a-08cd-4a50-9c10-e96df76b946e-iptables-alerter-script\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.762466 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.762395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l6z9\" (UniqueName: \"kubernetes.io/projected/2d951b3a-08cd-4a50-9c10-e96df76b946e-kube-api-access-2l6z9\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.762686 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.762644    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d951b3a-08cd-4a50-9c10-e96df76b946e-host-slash\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.763317 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.763245    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d951b3a-08cd-4a50-9c10-e96df76b946e-iptables-alerter-script\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.770485 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.770462    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l6z9\" (UniqueName: \"kubernetes.io/projected/2d951b3a-08cd-4a50-9c10-e96df76b946e-kube-api-access-2l6z9\") pod \"iptables-alerter-r7bbj\" (UID: \"2d951b3a-08cd-4a50-9c10-e96df76b946e\") " pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.846154 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.846065    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv"
Jul 06 01:20:02.854434 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.854406    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:02.855591 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.855563    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0c16fd_77bd_48c0_ad86_0e2ae810376f.slice/crio-b786db2e7dc83474801276c3cfba4fcf85400e2761f0fb2bd463fcc1a6ea1bb4 WatchSource:0}: Error finding container b786db2e7dc83474801276c3cfba4fcf85400e2761f0fb2bd463fcc1a6ea1bb4: Status 404 returned error can't find the container with id b786db2e7dc83474801276c3cfba4fcf85400e2761f0fb2bd463fcc1a6ea1bb4
Jul 06 01:20:02.862516 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.862495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b8zdg"
Jul 06 01:20:02.863810 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.863760    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f29e00_0b09_48e9_8218_e1885f613b85.slice/crio-e9dac1c6643506443dad02539208d94ebae67a785b2ea0f7d8f1cb25f64ac0c5 WatchSource:0}: Error finding container e9dac1c6643506443dad02539208d94ebae67a785b2ea0f7d8f1cb25f64ac0c5: Status 404 returned error can't find the container with id e9dac1c6643506443dad02539208d94ebae67a785b2ea0f7d8f1cb25f64ac0c5
Jul 06 01:20:02.870306 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.870165    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:02.871877 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.871548    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abdace0_a781_4cae_8d2b_62bb2ea4c4b0.slice/crio-1f832dd2547db2f79681bc107cab31974b936f2d415b2f6f91ec30014e42a1d2 WatchSource:0}: Error finding container 1f832dd2547db2f79681bc107cab31974b936f2d415b2f6f91ec30014e42a1d2: Status 404 returned error can't find the container with id 1f832dd2547db2f79681bc107cab31974b936f2d415b2f6f91ec30014e42a1d2
Jul 06 01:20:02.876822 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.876802    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tcksn"
Jul 06 01:20:02.879197 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.879170    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d676d2_25be_4f71_b759_f3e95c48d690.slice/crio-3e606f8e1de7d931685c64e39617877ea22b37affde477ca1102f0ad59d0ed9a WatchSource:0}: Error finding container 3e606f8e1de7d931685c64e39617877ea22b37affde477ca1102f0ad59d0ed9a: Status 404 returned error can't find the container with id 3e606f8e1de7d931685c64e39617877ea22b37affde477ca1102f0ad59d0ed9a
Jul 06 01:20:02.882924 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.882905    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26d9w"
Jul 06 01:20:02.884584 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.884562    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8373e415_2493_428c_b644_6d236f61f4ee.slice/crio-94c48c0fd6d33a2fb088dad9bd9ba391613a8815972890d196a5c53aa4217700 WatchSource:0}: Error finding container 94c48c0fd6d33a2fb088dad9bd9ba391613a8815972890d196a5c53aa4217700: Status 404 returned error can't find the container with id 94c48c0fd6d33a2fb088dad9bd9ba391613a8815972890d196a5c53aa4217700
Jul 06 01:20:02.891450 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.891430    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s6r8x"
Jul 06 01:20:02.893171 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.893107    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d794170_f755_4376_842b_17583c87c9c5.slice/crio-36c65cd65a8c5e50a9a650c833f9d18120cab4492405f7322afb02e21a13d5f3 WatchSource:0}: Error finding container 36c65cd65a8c5e50a9a650c833f9d18120cab4492405f7322afb02e21a13d5f3: Status 404 returned error can't find the container with id 36c65cd65a8c5e50a9a650c833f9d18120cab4492405f7322afb02e21a13d5f3
Jul 06 01:20:02.899285 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.899266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5smz7"
Jul 06 01:20:02.901562 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.901541    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0122480_e48e_449b_b4eb_2f78298552b4.slice/crio-07844d79faf73e159208ef59b1a63ca6a0a4c52a8b94abbb4bd78f973ef93caa WatchSource:0}: Error finding container 07844d79faf73e159208ef59b1a63ca6a0a4c52a8b94abbb4bd78f973ef93caa: Status 404 returned error can't find the container with id 07844d79faf73e159208ef59b1a63ca6a0a4c52a8b94abbb4bd78f973ef93caa
Jul 06 01:20:02.906422 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.906402    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7bbj"
Jul 06 01:20:02.911174 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.911152    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3802e17_f464_4bd6_b7ea_7ebfa91b5cf8.slice/crio-810d2049a21640a1e97e0682046ca2384c837ed70e054763dc01256ac0bd7b77 WatchSource:0}: Error finding container 810d2049a21640a1e97e0682046ca2384c837ed70e054763dc01256ac0bd7b77: Status 404 returned error can't find the container with id 810d2049a21640a1e97e0682046ca2384c837ed70e054763dc01256ac0bd7b77
Jul 06 01:20:02.917712 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:02.917688    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d951b3a_08cd_4a50_9c10_e96df76b946e.slice/crio-380edeedfeddab384ef42bb29116662f2d3016faf9319c91125fc186a9b45d9a WatchSource:0}: Error finding container 380edeedfeddab384ef42bb29116662f2d3016faf9319c91125fc186a9b45d9a: Status 404 returned error can't find the container with id 380edeedfeddab384ef42bb29116662f2d3016faf9319c91125fc186a9b45d9a
Jul 06 01:20:02.936270 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:02.936242    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 01:20:03.266441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.266370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:03.266441 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.266410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266508    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266522    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.266542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266558    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:04.266545771 +0000 UTC m=+4.111695162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266585    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:04.266567403 +0000 UTC m=+4.111716816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266640    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:03.266653 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266658    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:03.266925 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266670    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:03.266925 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:03.266713    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:04.266699053 +0000 UTC m=+4.111848447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:03.616773 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.616453    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:15:02 +0000 UTC" deadline="2028-03-09 04:33:03.647530592 +0000 UTC"
Jul 06 01:20:03.616773 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.616713    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14691h13m0.030824149s"
Jul 06 01:20:03.660330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.660210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s6r8x" event={"ID":"e0122480-e48e-449b-b4eb-2f78298552b4","Type":"ContainerStarted","Data":"07844d79faf73e159208ef59b1a63ca6a0a4c52a8b94abbb4bd78f973ef93caa"}
Jul 06 01:20:03.662079 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.662027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26d9w" event={"ID":"1d794170-f755-4376-842b-17583c87c9c5","Type":"ContainerStarted","Data":"36c65cd65a8c5e50a9a650c833f9d18120cab4492405f7322afb02e21a13d5f3"}
Jul 06 01:20:03.663728 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.663693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"3e606f8e1de7d931685c64e39617877ea22b37affde477ca1102f0ad59d0ed9a"}
Jul 06 01:20:03.665263 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.665193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-htk6z" event={"ID":"53f29e00-0b09-48e9-8218-e1885f613b85","Type":"ContainerStarted","Data":"e9dac1c6643506443dad02539208d94ebae67a785b2ea0f7d8f1cb25f64ac0c5"}
Jul 06 01:20:03.667468 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.667437    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv" event={"ID":"da0c16fd-77bd-48c0-ad86-0e2ae810376f","Type":"ContainerStarted","Data":"b786db2e7dc83474801276c3cfba4fcf85400e2761f0fb2bd463fcc1a6ea1bb4"}
Jul 06 01:20:03.670697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.670674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7bbj" event={"ID":"2d951b3a-08cd-4a50-9c10-e96df76b946e","Type":"ContainerStarted","Data":"380edeedfeddab384ef42bb29116662f2d3016faf9319c91125fc186a9b45d9a"}
Jul 06 01:20:03.672304 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.672278    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tcksn" event={"ID":"8373e415-2493-428c-b644-6d236f61f4ee","Type":"ContainerStarted","Data":"94c48c0fd6d33a2fb088dad9bd9ba391613a8815972890d196a5c53aa4217700"}
Jul 06 01:20:03.673638 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.673596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b8zdg" event={"ID":"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0","Type":"ContainerStarted","Data":"1f832dd2547db2f79681bc107cab31974b936f2d415b2f6f91ec30014e42a1d2"}
Jul 06 01:20:03.675405 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:03.675364    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerStarted","Data":"810d2049a21640a1e97e0682046ca2384c837ed70e054763dc01256ac0bd7b77"}
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.275680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.275731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.275768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.275906    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.275923    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.275936    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.275992    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:06.275975057 +0000 UTC m=+6.121124464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.276439    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.276496    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:06.276475779 +0000 UTC m=+6.121625175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.276571    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:04.276644 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.276613    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:06.276591149 +0000 UTC m=+6.121740546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:04.617431 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.617336    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:15:02 +0000 UTC" deadline="2028-04-01 14:12:11.099027174 +0000 UTC"
Jul 06 01:20:04.617431 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.617374    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15252h52m6.481657514s"
Jul 06 01:20:04.653314 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.653250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:04.653474 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.653383    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:04.653792 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.653772    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:04.653936 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.653872    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:04.654004 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:04.653968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:04.654078 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:04.654059    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:05.684163 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:05.684100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal" event={"ID":"c581f67582f2b7493459f83b250c1488","Type":"ContainerStarted","Data":"91629e1f381581c78f6db6c5f0a2ea3e6a137676271bb6223b78af0d0eb20684"}
Jul 06 01:20:06.292430 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.292387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:06.292610 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.292440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:06.292610 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.292480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:06.292727 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.292617    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:06.292727 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.292639    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:06.292727 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.292652    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:06.292727 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.292706    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:10.292687225 +0000 UTC m=+10.137836618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:06.293140 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.293092    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:06.293221 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.293160    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:10.2931429 +0000 UTC m=+10.138292296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:06.293221 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.293218    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:06.293328 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.293254    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:10.293242767 +0000 UTC m=+10.138392160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:06.652150 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.652064    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:06.652314 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.652202    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:06.652381 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.652365    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:06.652631 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.652439    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:06.652631 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.652486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:06.652631 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:06.652534    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:06.688461 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.688422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal" event={"ID":"176a852dc8aa3e063105495696527f75","Type":"ContainerStarted","Data":"6ca02314d645879b19211e37c09b5b1e64a50025c45303f99d81b1fe856184b4"}
Jul 06 01:20:06.703141 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:06.703070    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-137.ec2.internal" podStartSLOduration=5.703051954 podStartE2EDuration="5.703051954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:05.699944002 +0000 UTC m=+5.545093431" watchObservedRunningTime="2026-07-06 01:20:06.703051954 +0000 UTC m=+6.548201369"
Jul 06 01:20:07.694222 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:07.693696    2567 generic.go:356] "Generic (PLEG): container finished" podID="176a852dc8aa3e063105495696527f75" containerID="6ca02314d645879b19211e37c09b5b1e64a50025c45303f99d81b1fe856184b4" exitCode=0
Jul 06 01:20:07.694222 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:07.693739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal" event={"ID":"176a852dc8aa3e063105495696527f75","Type":"ContainerDied","Data":"6ca02314d645879b19211e37c09b5b1e64a50025c45303f99d81b1fe856184b4"}
Jul 06 01:20:08.652204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:08.651437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:08.652204 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:08.651575    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:08.652204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:08.651985    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:08.652204 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:08.652114    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:08.652723 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:08.652579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:08.652723 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:08.652674    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:10.328678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.328637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.328697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.328730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.328854    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.328871    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.328882    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:10.329141 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.328942    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:18.328924023 +0000 UTC m=+18.174073432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:10.329484 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.329327    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:10.329484 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.329364    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:18.329353076 +0000 UTC m=+18.174502466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:10.329484 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.329406    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:10.329484 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.329433    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:18.329422858 +0000 UTC m=+18.174572247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:10.653100 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.653031    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:10.653245 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.653188    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:10.653587 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.653570    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:10.653689 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.653664    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:10.655395 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:10.655374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:10.655504 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:10.655474    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:12.650673 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:12.650640    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:12.650673 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:12.650674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:12.651223 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:12.650640    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:12.651223 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:12.650771    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:12.651223 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:12.650860    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:12.651223 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:12.650964    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:14.651070 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:14.651037    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:14.651563 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:14.651037    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:14.651563 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:14.651190    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:14.651563 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:14.651245    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:14.651563 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:14.651390    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:14.651563 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:14.651495    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:16.650518 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:16.650482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:16.650518 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:16.650482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:16.650970 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:16.650630    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:16.651023 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:16.650995    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:16.651134 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:16.651104    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:16.651252 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:16.651200    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:18.384082 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.384048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.384100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.384147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384193    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384235    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384272    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:34.384246431 +0000 UTC m=+34.229395826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384297    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:34.384279549 +0000 UTC m=+34.229428942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384299    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384318    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384330    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:18.384523 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.384374    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:20:34.384364987 +0000 UTC m=+34.229514377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:18.650446 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.650359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:18.650446 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.650391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:18.650640 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:18.650359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:18.650640 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.650507    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:18.650640 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.650534    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:18.650884 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:18.650855    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:20.651592 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:20.651560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:20.652009 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:20.651595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:20.652009 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:20.651653    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:20.652009 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:20.651736    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:20.652009 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:20.651832    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:20.652009 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:20.651908    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.650943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.650942    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:22.651180    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.650988    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:22.651225    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:22.656698 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:22.651288    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:22.723484 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.723462    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:20:22.723894 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.723865    2567 generic.go:356] "Generic (PLEG): container finished" podID="60d676d2-25be-4f71-b759-f3e95c48d690" containerID="e8177214e136e0beb4054e5154d09643656d4b61344c7bf3fee420390ba493f7" exitCode=1
Jul 06 01:20:22.724002 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.723913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerDied","Data":"e8177214e136e0beb4054e5154d09643656d4b61344c7bf3fee420390ba493f7"}
Jul 06 01:20:22.724063 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.724006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"9231c9c3100a0f67bbad7fb4ec34c95fcffa8db23f8665dd3c9ccaec7d9ef056"}
Jul 06 01:20:22.725931 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.725503    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-htk6z" event={"ID":"53f29e00-0b09-48e9-8218-e1885f613b85","Type":"ContainerStarted","Data":"3866dc144b228f922b8dd90e8a693213686f30ca96bdeb8bdd2ab1d311e23614"}
Jul 06 01:20:22.727076 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.727051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv" event={"ID":"da0c16fd-77bd-48c0-ad86-0e2ae810376f","Type":"ContainerStarted","Data":"a240379e4b8f1d2ba1e8c4062ce550815536afa7712cbf09b7a749884d0c228a"}
Jul 06 01:20:22.728468 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.728438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tcksn" event={"ID":"8373e415-2493-428c-b644-6d236f61f4ee","Type":"ContainerStarted","Data":"ceeb1005d97e8e789520034970cb6954a00a5e16b5e9269476c4fcf1c282c42b"}
Jul 06 01:20:22.730447 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.730415    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b8zdg" event={"ID":"9abdace0-a781-4cae-8d2b-62bb2ea4c4b0","Type":"ContainerStarted","Data":"c34629b915130a3ec6efa839f62ed104f423c116e76de96f2977c4c2c9603ca1"}
Jul 06 01:20:22.732946 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.732912    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal" event={"ID":"176a852dc8aa3e063105495696527f75","Type":"ContainerStarted","Data":"fde131c9f0e697b3af5d6d1ff78169fa7325c84d3f7522487cbc180cd71d106e"}
Jul 06 01:20:22.734507 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.734487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerStarted","Data":"7e11c71877621807d6d6e3d6038563be9753b85bd0848f627605120fd68c999e"}
Jul 06 01:20:22.736340 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.736300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s6r8x" event={"ID":"e0122480-e48e-449b-b4eb-2f78298552b4","Type":"ContainerStarted","Data":"8ffdb4aac40e99c3ee18e5ad0ab0ad93c38464eef77d2d35b098f9f61d3d6aa4"}
Jul 06 01:20:22.737984 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.737955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26d9w" event={"ID":"1d794170-f755-4376-842b-17583c87c9c5","Type":"ContainerStarted","Data":"81710deff2c69c51f1e387ea8a320b754b1fea871ad04e4a39cf7c8760b0bb67"}
Jul 06 01:20:22.740019 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.739978    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-htk6z" podStartSLOduration=7.605075448 podStartE2EDuration="22.739967436s" totalImagesPullingTime="15.134891988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.739865869 +0000 UTC m=+22.585015315" watchObservedRunningTime="2026-07-06 01:20:22.739967436 +0000 UTC m=+22.585116848"
Jul 06 01:20:22.758322 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.758280    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tcksn" podStartSLOduration=3.330302603 podStartE2EDuration="22.758260464s" totalImagesPullingTime="19.427957861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.757877198 +0000 UTC m=+22.603026610" watchObservedRunningTime="2026-07-06 01:20:22.758260464 +0000 UTC m=+22.603409875"
Jul 06 01:20:22.771718 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.771672    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-26d9w" podStartSLOduration=3.434345825 podStartE2EDuration="22.771656185s" totalImagesPullingTime="19.33731036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.771388555 +0000 UTC m=+22.616537991" watchObservedRunningTime="2026-07-06 01:20:22.771656185 +0000 UTC m=+22.616805597"
Jul 06 01:20:22.786848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.786801    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-137.ec2.internal" podStartSLOduration=21.786786605 podStartE2EDuration="21.786786605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.786520211 +0000 UTC m=+22.631669624" watchObservedRunningTime="2026-07-06 01:20:22.786786605 +0000 UTC m=+22.631936018"
Jul 06 01:20:22.807572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.805572    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b8zdg" podStartSLOduration=3.42230255 podStartE2EDuration="22.805550641s" totalImagesPullingTime="19.383248091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.804908395 +0000 UTC m=+22.650057807" watchObservedRunningTime="2026-07-06 01:20:22.805550641 +0000 UTC m=+22.650700053"
Jul 06 01:20:22.820950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:22.820906    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s6r8x" podStartSLOduration=2.830200783 podStartE2EDuration="21.820892295s" totalImagesPullingTime="18.990691512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:22.820670814 +0000 UTC m=+22.665820226" watchObservedRunningTime="2026-07-06 01:20:22.820892295 +0000 UTC m=+22.666041707"
Jul 06 01:20:23.742289 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.742002    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7bbj" event={"ID":"2d951b3a-08cd-4a50-9c10-e96df76b946e","Type":"ContainerStarted","Data":"58a297379758fb6864d79c2f05b0070a8fd71759bfa2dfa2b64cee9e21b975d2"}
Jul 06 01:20:23.743553 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.743529    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="7e11c71877621807d6d6e3d6038563be9753b85bd0848f627605120fd68c999e" exitCode=0
Jul 06 01:20:23.743665 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.743611    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"7e11c71877621807d6d6e3d6038563be9753b85bd0848f627605120fd68c999e"}
Jul 06 01:20:23.746136 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.746105    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:20:23.746835 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.746819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"48458183b0e76df64ede3ee6bc75d71855527f735580130a729715415e8fb3ae"}
Jul 06 01:20:23.746924 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.746909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"2ced68fe10a6b535949490afb50f09a17cf343a586ab148c9a4a4fe9799e7713"}
Jul 06 01:20:23.746963 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.746948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"33110a3c61f0ac5c4ed582c64b16e9d34df802882792d5a97f8d93d21a89e55b"}
Jul 06 01:20:23.746995 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.746976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"33f7930df06b2f5e3b2e86511ae8308ea6b63792200a381bcd6ca12382fa974e"}
Jul 06 01:20:23.758672 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:23.758634    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r7bbj" podStartSLOduration=3.780371374 podStartE2EDuration="22.758623521s" totalImagesPullingTime="18.978252147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:23.758139664 +0000 UTC m=+23.603289101" watchObservedRunningTime="2026-07-06 01:20:23.758623521 +0000 UTC m=+23.603772933"
Jul 06 01:20:24.134105 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.134083    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 01:20:24.619881 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.619769    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T01:20:24.134098477Z","UUID":"27b6f774-293c-4478-9db8-1869adbbcd60","Handler":null,"Name":"","Endpoint":""}
Jul 06 01:20:24.622195 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.622169    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 01:20:24.622321 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.622217    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 01:20:24.650493 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.650424    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:24.650640 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:24.650550    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:24.650640 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.650562    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:24.650640 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:24.650626    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:24.650837 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.650648    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:24.650837 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:24.650695    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:24.751094 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:24.751059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv" event={"ID":"da0c16fd-77bd-48c0-ad86-0e2ae810376f","Type":"ContainerStarted","Data":"9d0b86593a66016cc1fcdf8672eeb1a23e98aa5fb73dc9a394167df684127f79"}
Jul 06 01:20:25.348201 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.347955    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:25.348603 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.348576    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:25.756135 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.756107    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:20:25.756553 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.756530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"06470fdbc9ea6691d9856645ff6258065ce7e5551e8948899bbe5905a55034cc"}
Jul 06 01:20:25.759045 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.759019    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv" event={"ID":"da0c16fd-77bd-48c0-ad86-0e2ae810376f","Type":"ContainerStarted","Data":"546c0ca72ee49f7a05892fc13f80f4b8b1a7ea1cba043ce6dede866bcd03bcc6"}
Jul 06 01:20:25.759386 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.759370    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:25.759708 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.759690    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-htk6z"
Jul 06 01:20:25.774592 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:25.774550    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4cfv" podStartSLOduration=3.474113052 podStartE2EDuration="25.774536026s" totalImagesPullingTime="22.300422974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:25.77408909 +0000 UTC m=+25.619238522" watchObservedRunningTime="2026-07-06 01:20:25.774536026 +0000 UTC m=+25.619685437"
Jul 06 01:20:26.654280 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:26.654251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:26.654460 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:26.654360    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:26.654460 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:26.654251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:26.654460 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:26.654436    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:26.654460 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:26.654251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:26.654657 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:26.654484    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:28.650786 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.650751    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:28.651678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.650751    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:28.651678 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:28.650882    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:28.651678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.650892    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:28.651678 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:28.650975    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:28.651678 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:28.651077    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:28.772689 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.772467    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:20:28.773157 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.773112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"92fb016b2f0de4c429e5069f99d3a9431dee0efbcbab22a19a6e05f6adba59de"}
Jul 06 01:20:28.775307 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.773613    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:28.775307 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.773709    2567 scope.go:117] "RemoveContainer" containerID="e8177214e136e0beb4054e5154d09643656d4b61344c7bf3fee420390ba493f7"
Jul 06 01:20:28.775307 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.773777    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:28.775307 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.773910    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:28.796679 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.795291    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:28.796679 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:28.796022    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:20:29.779547 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:29.779512    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="3c8e18949bd2b7c1d369b9b56e10ce1451c876fc24400e86c0e09d6653c90094" exitCode=0
Jul 06 01:20:29.780152 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:29.779593    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"3c8e18949bd2b7c1d369b9b56e10ce1451c876fc24400e86c0e09d6653c90094"}
Jul 06 01:20:29.783665 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:29.783647    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:20:29.783958 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:29.783941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" event={"ID":"60d676d2-25be-4f71-b759-f3e95c48d690","Type":"ContainerStarted","Data":"39f6decff5c733ec63cdd855ef30eac3f0eec5685fdec5716fdf4dd046ad8e2a"}
Jul 06 01:20:29.823604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:29.823550    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd" podStartSLOduration=10.420651292 podStartE2EDuration="29.823535453s" totalImagesPullingTime="19.402884161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:29.8226426 +0000 UTC m=+29.667792022" watchObservedRunningTime="2026-07-06 01:20:29.823535453 +0000 UTC m=+29.668684864"
Jul 06 01:20:30.570474 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.570445    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gc54p"]
Jul 06 01:20:30.570693 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.570578    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:30.570693 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:30.570676    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:30.573983 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.573957    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ktvvs"]
Jul 06 01:20:30.574110 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.574075    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:30.574225 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:30.574205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:30.574480 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.574464    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlnzs"]
Jul 06 01:20:30.574547 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.574538    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:30.574625 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:30.574610    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:30.788901 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.788685    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="b5bc1ca275caaf3d208431faf3941b78c385ecb78a2c52493c6eb3984e05d4c4" exitCode=0
Jul 06 01:20:30.789270 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:30.788731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"b5bc1ca275caaf3d208431faf3941b78c385ecb78a2c52493c6eb3984e05d4c4"}
Jul 06 01:20:31.650697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:31.650635    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:31.650810 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:31.650767    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:31.793724 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:31.793696    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="bdc7ad94c2a707bc1a407f5033d08dc51ef0afb63b7842c294c6d59f15c4807b" exitCode=0
Jul 06 01:20:31.794157 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:31.793750    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"bdc7ad94c2a707bc1a407f5033d08dc51ef0afb63b7842c294c6d59f15c4807b"}
Jul 06 01:20:32.650851 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:32.650821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:32.650851 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:32.650834    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:32.651063 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:32.650949    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:32.651105 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:32.651082    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:33.650408 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:33.650374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:33.650853 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:33.650474    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlnzs" podUID="54e15649-83ed-4352-8a50-349ee45263eb"
Jul 06 01:20:34.398849 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:34.398814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:34.398867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:34.398895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.398982    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.398986    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399008    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399020    2567 projected.go:196] Error preparing data for projected volume kube-api-access-h8vvx for pod openshift-network-diagnostics/network-check-target-xlnzs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:34.399046 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399038    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret podName:0c277b2c-503f-48a2-9429-6ad2c2094008 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:06.399021033 +0000 UTC m=+66.244170425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret") pod "global-pull-secret-syncer-gc54p" (UID: "0c277b2c-503f-48a2-9429-6ad2c2094008") : object "kube-system"/"original-pull-secret" not registered
Jul 06 01:20:34.399435 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399053    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx podName:54e15649-83ed-4352-8a50-349ee45263eb nodeName:}" failed. No retries permitted until 2026-07-06 01:21:06.399043802 +0000 UTC m=+66.244193192 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8vvx" (UniqueName: "kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx") pod "network-check-target-xlnzs" (UID: "54e15649-83ed-4352-8a50-349ee45263eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 01:20:34.399435 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399096    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:34.399435 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.399146    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs podName:7bc9f526-fc74-4399-95db-9cd060968c31 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:06.399115887 +0000 UTC m=+66.244265284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs") pod "network-metrics-daemon-ktvvs" (UID: "7bc9f526-fc74-4399-95db-9cd060968c31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 01:20:34.651090 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:34.651021    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:34.651090 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:34.651057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:34.651448 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.651173    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gc54p" podUID="0c277b2c-503f-48a2-9429-6ad2c2094008"
Jul 06 01:20:34.651448 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:34.651279    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ktvvs" podUID="7bc9f526-fc74-4399-95db-9cd060968c31"
Jul 06 01:20:35.432305 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.432273    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-137.ec2.internal" event="NodeReady"
Jul 06 01:20:35.432483 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.432435    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 01:20:35.468579 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.468550    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"]
Jul 06 01:20:35.492657 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.492630    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:20:35.493700 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.492902    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.496509 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.496486    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8hj88\""
Jul 06 01:20:35.499232 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.499209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 06 01:20:35.500043 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.500017    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 06 01:20:35.503275 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.503256    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 06 01:20:35.508183 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.508163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"]
Jul 06 01:20:35.508183 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.508191    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:20:35.508326 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.508210    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"]
Jul 06 01:20:35.508326 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.508272    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.508589 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.508572    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 06 01:20:35.511286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.511267    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 01:20:35.511602 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.511585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 01:20:35.512305 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.512282    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qv997\""
Jul 06 01:20:35.513966 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.512613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 01:20:35.523114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.523079    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 01:20:35.524274 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.524251    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"]
Jul 06 01:20:35.524474 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.524430    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.527083 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.527061    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hnpvc\""
Jul 06 01:20:35.527879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.527301    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 01:20:35.527879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.527523    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 01:20:35.544323 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.544302    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ctfzm"]
Jul 06 01:20:35.544456 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.544441    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"
Jul 06 01:20:35.546991 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.546970    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 01:20:35.547084 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.546972    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7t2dq\""
Jul 06 01:20:35.547084 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.546997    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 01:20:35.562408 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.562392    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"]
Jul 06 01:20:35.562506 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.562492    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.564897 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.564876    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 01:20:35.565008 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.564886    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 01:20:35.565008 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.564998    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b9srw\""
Jul 06 01:20:35.565161 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.564998    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 01:20:35.581461 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.581434    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"]
Jul 06 01:20:35.581570 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.581552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.584345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.584050    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 01:20:35.584345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.584135    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 01:20:35.584345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.584167    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 01:20:35.584345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.584230    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 01:20:35.584345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.584137    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5wpc5\""
Jul 06 01:20:35.595546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.595529    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"]
Jul 06 01:20:35.595664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.595650    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.597824 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.597802    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 01:20:35.597984 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.597969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 01:20:35.598056 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.598033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 01:20:35.598259 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.598245    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 01:20:35.607995 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.607978    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608072 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42rdx\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608072 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608049    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608203 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75x97\" (UniqueName: \"kubernetes.io/projected/d0c5f528-2201-4f1a-b0b8-4a40108c3db1-kube-api-access-75x97\") pod \"network-check-source-989f8b8dc-7qs55\" (UID: \"d0c5f528-2201-4f1a-b0b8-4a40108c3db1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"
Jul 06 01:20:35.608203 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.608290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1018a839-0ecd-4609-900f-396112d157a1-config\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.608290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608299    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1018a839-0ecd-4609-900f-396112d157a1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.608420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64g9\" (UniqueName: \"kubernetes.io/projected/1018a839-0ecd-4609-900f-396112d157a1-kube-api-access-k64g9\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.608615 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608456    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.608615 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.608483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.610494 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.609564    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w6cs8"]
Jul 06 01:20:35.610494 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.609685    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.611895 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.611878    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627806    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ctfzm"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627863    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627906    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627918    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.627922    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"]
Jul 06 01:20:35.628604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.628323    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6cs8"]
Jul 06 01:20:35.630085 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.630059    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 01:20:35.630198 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.630103    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 01:20:35.630253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.630220    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g4lpx\""
Jul 06 01:20:35.651134 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.651105    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:20:35.653521 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.653502    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kdr7\""
Jul 06 01:20:35.709507 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr8tl\" (UniqueName: \"kubernetes.io/projected/776883bd-0078-48bc-adc7-06d04c2d0921-kube-api-access-sr8tl\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.709507 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709476    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngt9x\" (UniqueName: \"kubernetes.io/projected/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-kube-api-access-ngt9x\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.709680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1018a839-0ecd-4609-900f-396112d157a1-config\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.709680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.709680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv5lb\" (UniqueName: \"kubernetes.io/projected/6c131c57-4eb6-45fd-9dd8-76c09f647891-kube-api-access-tv5lb\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.709680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64g9\" (UniqueName: \"kubernetes.io/projected/1018a839-0ecd-4609-900f-396112d157a1-kube-api-access-k64g9\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c306948-74cd-4b86-9383-c2687d0503c5-config-volume\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709784    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42rdx\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsk6b\" (UniqueName: \"kubernetes.io/projected/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-kube-api-access-vsk6b\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.709871 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.709864    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.709881    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mzr5\" (UniqueName: \"kubernetes.io/projected/1c306948-74cd-4b86-9383-c2687d0503c5-kube-api-access-4mzr5\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75x97\" (UniqueName: \"kubernetes.io/projected/d0c5f528-2201-4f1a-b0b8-4a40108c3db1-kube-api-access-75x97\") pod \"network-check-source-989f8b8dc-7qs55\" (UID: \"d0c5f528-2201-4f1a-b0b8-4a40108c3db1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.709991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/776883bd-0078-48bc-adc7-06d04c2d0921-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710065    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1018a839-0ecd-4609-900f-396112d157a1-config\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.710096    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:36.210080424 +0000 UTC m=+36.055229814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710223    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.710260 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.710242    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.710291    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:36.210273937 +0000 UTC m=+36.055423326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710320    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c131c57-4eb6-45fd-9dd8-76c09f647891-cert\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710380    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-tmp\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1018a839-0ecd-4609-900f-396112d157a1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710637    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.710829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710757    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c306948-74cd-4b86-9383-c2687d0503c5-tmp-dir\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.711282 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.710877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-klusterlet-config\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.711282 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.711112    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.711739 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.711721    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.715194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.715174    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.715275 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.715195    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.715324 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.715303    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1018a839-0ecd-4609-900f-396112d157a1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.718577 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.718553    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.718970 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.718949    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42rdx\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:35.719361 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.719338    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64g9\" (UniqueName: \"kubernetes.io/projected/1018a839-0ecd-4609-900f-396112d157a1-kube-api-access-k64g9\") pod \"service-ca-operator-5d46d9ffd4-z7rjt\" (UID: \"1018a839-0ecd-4609-900f-396112d157a1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.719549 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.719530    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75x97\" (UniqueName: \"kubernetes.io/projected/d0c5f528-2201-4f1a-b0b8-4a40108c3db1-kube-api-access-75x97\") pod \"network-check-source-989f8b8dc-7qs55\" (UID: \"d0c5f528-2201-4f1a-b0b8-4a40108c3db1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"
Jul 06 01:20:35.805301 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.805276    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"
Jul 06 01:20:35.811607 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsk6b\" (UniqueName: \"kubernetes.io/projected/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-kube-api-access-vsk6b\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.811714 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mzr5\" (UniqueName: \"kubernetes.io/projected/1c306948-74cd-4b86-9383-c2687d0503c5-kube-api-access-4mzr5\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.811714 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.811714 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/776883bd-0078-48bc-adc7-06d04c2d0921-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.811879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.811879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.811879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c131c57-4eb6-45fd-9dd8-76c09f647891-cert\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.811879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-tmp\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.811879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c306948-74cd-4b86-9383-c2687d0503c5-tmp-dir\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-klusterlet-config\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.811979    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr8tl\" (UniqueName: \"kubernetes.io/projected/776883bd-0078-48bc-adc7-06d04c2d0921-kube-api-access-sr8tl\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.812007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngt9x\" (UniqueName: \"kubernetes.io/projected/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-kube-api-access-ngt9x\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.812041    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.812104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.812071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv5lb\" (UniqueName: \"kubernetes.io/projected/6c131c57-4eb6-45fd-9dd8-76c09f647891-kube-api-access-tv5lb\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.812619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.812155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c306948-74cd-4b86-9383-c2687d0503c5-config-volume\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.812619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.812444    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-tmp\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.813045 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.813021    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c306948-74cd-4b86-9383-c2687d0503c5-config-volume\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.813148 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.813109    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c306948-74cd-4b86-9383-c2687d0503c5-tmp-dir\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.813894 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.813867    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.814942 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.814917    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c131c57-4eb6-45fd-9dd8-76c09f647891-cert\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.815038 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.815012    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.815038 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.815017    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:35.815191 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.815059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-hub\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.815191 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:35.815096    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:36.315077916 +0000 UTC m=+36.160227306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:20:35.815717 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.815689    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-ca\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.815895 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.815873    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/776883bd-0078-48bc-adc7-06d04c2d0921-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.816351 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.816327    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-klusterlet-config\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.816725 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.816704    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.820510 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.820486    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr8tl\" (UniqueName: \"kubernetes.io/projected/776883bd-0078-48bc-adc7-06d04c2d0921-kube-api-access-sr8tl\") pod \"managed-serviceaccount-addon-agent-b5897d64b-t625q\" (UID: \"776883bd-0078-48bc-adc7-06d04c2d0921\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.821064 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.821042    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngt9x\" (UniqueName: \"kubernetes.io/projected/4f9f8db7-d5b6-456c-8f37-7237ff3384ab-kube-api-access-ngt9x\") pod \"klusterlet-addon-workmgr-6457ff646d-vxkff\" (UID: \"4f9f8db7-d5b6-456c-8f37-7237ff3384ab\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:35.821937 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.821894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv5lb\" (UniqueName: \"kubernetes.io/projected/6c131c57-4eb6-45fd-9dd8-76c09f647891-kube-api-access-tv5lb\") pod \"ingress-canary-ctfzm\" (UID: \"6c131c57-4eb6-45fd-9dd8-76c09f647891\") " pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.822665 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.822640    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mzr5\" (UniqueName: \"kubernetes.io/projected/1c306948-74cd-4b86-9383-c2687d0503c5-kube-api-access-4mzr5\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:35.823547 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.823529    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsk6b\" (UniqueName: \"kubernetes.io/projected/7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9-kube-api-access-vsk6b\") pod \"cluster-proxy-proxy-agent-68f748fcc6-rt7fn\" (UID: \"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.854162 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.854106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"
Jul 06 01:20:35.872029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.872000    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ctfzm"
Jul 06 01:20:35.901450 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.901429    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"
Jul 06 01:20:35.909616 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.909592    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:20:35.936815 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:35.936705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:36.215253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.215221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:36.215401 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.215265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:36.215401 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.215366    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:36.215401 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.215374    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:36.215401 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.215398    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:36.215536 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.215423    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.21540923 +0000 UTC m=+37.060558619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:36.215536 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.215459    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.215447056 +0000 UTC m=+37.060596446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:36.316888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.316568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:36.317062 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.316706    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:36.317062 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:36.317001    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.316977162 +0000 UTC m=+37.162126553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:20:36.491842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.491762    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"]
Jul 06 01:20:36.495988 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.495966    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt"]
Jul 06 01:20:36.499171 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.499151    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55"]
Jul 06 01:20:36.501945 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.501919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q"]
Jul 06 01:20:36.513613 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.513593    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ctfzm"]
Jul 06 01:20:36.516588 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.516568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"]
Jul 06 01:20:36.586169 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:36.586143    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9f8db7_d5b6_456c_8f37_7237ff3384ab.slice/crio-fe4ccec4e6236cd29a22435671d3f117f6f7c30974be6f83614b0606a6eb5fb7 WatchSource:0}: Error finding container fe4ccec4e6236cd29a22435671d3f117f6f7c30974be6f83614b0606a6eb5fb7: Status 404 returned error can't find the container with id fe4ccec4e6236cd29a22435671d3f117f6f7c30974be6f83614b0606a6eb5fb7
Jul 06 01:20:36.586684 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:36.586665    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1018a839_0ecd_4609_900f_396112d157a1.slice/crio-c89968732513601d1b15a215fed5cb72f0635bffde5e2dd343a13e87ccd1980b WatchSource:0}: Error finding container c89968732513601d1b15a215fed5cb72f0635bffde5e2dd343a13e87ccd1980b: Status 404 returned error can't find the container with id c89968732513601d1b15a215fed5cb72f0635bffde5e2dd343a13e87ccd1980b
Jul 06 01:20:36.587695 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:36.587671    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c5f528_2201_4f1a_b0b8_4a40108c3db1.slice/crio-2f75ebe46f682ce2602127174808f925eaea013ecc2ad3f35eb2da80e43a0c2a WatchSource:0}: Error finding container 2f75ebe46f682ce2602127174808f925eaea013ecc2ad3f35eb2da80e43a0c2a: Status 404 returned error can't find the container with id 2f75ebe46f682ce2602127174808f925eaea013ecc2ad3f35eb2da80e43a0c2a
Jul 06 01:20:36.588601 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:20:36.588453    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod776883bd_0078_48bc_adc7_06d04c2d0921.slice/crio-1a7116bf544cfd6ffd4b283bc4265442ab81f284cde9e33140d2ad305a95972f WatchSource:0}: Error finding container 1a7116bf544cfd6ffd4b283bc4265442ab81f284cde9e33140d2ad305a95972f: Status 404 returned error can't find the container with id 1a7116bf544cfd6ffd4b283bc4265442ab81f284cde9e33140d2ad305a95972f
Jul 06 01:20:36.650841 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.650819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:20:36.651017 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.650999    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:20:36.653203 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.653187    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bqkt\""
Jul 06 01:20:36.653203 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.653199    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 01:20:36.653576 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.653203    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 01:20:36.810344 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.810294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55" event={"ID":"d0c5f528-2201-4f1a-b0b8-4a40108c3db1","Type":"ContainerStarted","Data":"2f75ebe46f682ce2602127174808f925eaea013ecc2ad3f35eb2da80e43a0c2a"}
Jul 06 01:20:36.811542 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.811519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt" event={"ID":"1018a839-0ecd-4609-900f-396112d157a1","Type":"ContainerStarted","Data":"c89968732513601d1b15a215fed5cb72f0635bffde5e2dd343a13e87ccd1980b"}
Jul 06 01:20:36.814829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.814799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerStarted","Data":"b9d593196a6b80df61b0ac09028f8e21c5d0937aeb99e974ef1c9c5d26dcb362"}
Jul 06 01:20:36.815963 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.815938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff" event={"ID":"4f9f8db7-d5b6-456c-8f37-7237ff3384ab","Type":"ContainerStarted","Data":"fe4ccec4e6236cd29a22435671d3f117f6f7c30974be6f83614b0606a6eb5fb7"}
Jul 06 01:20:36.817165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.817144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q" event={"ID":"776883bd-0078-48bc-adc7-06d04c2d0921","Type":"ContainerStarted","Data":"1a7116bf544cfd6ffd4b283bc4265442ab81f284cde9e33140d2ad305a95972f"}
Jul 06 01:20:36.818350 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.818324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerStarted","Data":"5fe1abd3acfce8fbc26101dbbc098d1f9b067b760f75fd3e34c410f2f3042477"}
Jul 06 01:20:36.819537 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:36.819516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ctfzm" event={"ID":"6c131c57-4eb6-45fd-9dd8-76c09f647891","Type":"ContainerStarted","Data":"1fdd3314664c14cbc3c606a3f61905bf5327e8a511cdb6c3524d098fa40f13cc"}
Jul 06 01:20:37.225752 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:37.225714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:37.225916 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:37.225859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:37.225993 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.225976    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:37.226070 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.226057    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:39.226037341 +0000 UTC m=+39.071186737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:37.226152 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.225979    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:37.226152 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.226106    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:37.226254 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.226191    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:39.226170776 +0000 UTC m=+39.071320170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:37.327181 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:37.326715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:37.327181 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.326857    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:37.327181 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:37.326902    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:39.326889781 +0000 UTC m=+39.172039171 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:20:37.835148 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:37.834848    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="b9d593196a6b80df61b0ac09028f8e21c5d0937aeb99e974ef1c9c5d26dcb362" exitCode=0
Jul 06 01:20:37.835148 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:37.834925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"b9d593196a6b80df61b0ac09028f8e21c5d0937aeb99e974ef1c9c5d26dcb362"}
Jul 06 01:20:38.858860 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:38.858511    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8" containerID="266b769c88bf4895afabbd884ab678f09ff52174824dc36fb460fdccef9d5a35" exitCode=0
Jul 06 01:20:38.859345 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:38.858892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerDied","Data":"266b769c88bf4895afabbd884ab678f09ff52174824dc36fb460fdccef9d5a35"}
Jul 06 01:20:39.246420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:39.246386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:39.246613 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:39.246459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:39.246680 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.246667    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:39.246748 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.246732    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:43.246712392 +0000 UTC m=+43.091861806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:39.247208 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.247188    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:39.247208 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.247209    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:39.247366 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.247252    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:43.247237321 +0000 UTC m=+43.092386712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:39.350155 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:39.347302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:39.350155 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.347576    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:39.350155 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:39.347662    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:43.347641832 +0000 UTC m=+43.192791247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:20:39.868959 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:39.867812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5smz7" event={"ID":"f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8","Type":"ContainerStarted","Data":"a0dcc7fb7604d100c42be67fba2221e097663c31bf0e0f0be80dc59fa7cf0f15"}
Jul 06 01:20:39.891080 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:39.891019    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5smz7" podStartSLOduration=8.241174305 podStartE2EDuration="38.890998353s" totalImagesPullingTime="30.649824048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:39.889204493 +0000 UTC m=+39.734353929" watchObservedRunningTime="2026-07-06 01:20:39.890998353 +0000 UTC m=+39.736147768"
Jul 06 01:20:43.284262 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:43.284221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:43.284276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.284392    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.284413    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.284444    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.284477    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:51.284454968 +0000 UTC m=+51.129604377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:43.284887 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.284504    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:51.284485211 +0000 UTC m=+51.129634627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:43.385322 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:43.385284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:43.385512 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.385475    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:43.385563 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:43.385530    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:51.385512408 +0000 UTC m=+51.230661816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:20:47.891823 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.891572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ctfzm" event={"ID":"6c131c57-4eb6-45fd-9dd8-76c09f647891","Type":"ContainerStarted","Data":"baca3c172a4819926d8aa2c7e052fc8e1f179bed223b26050e86708ed6b01eb1"}
Jul 06 01:20:47.894078 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.893797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55" event={"ID":"d0c5f528-2201-4f1a-b0b8-4a40108c3db1","Type":"ContainerStarted","Data":"27d2330c258fada863c6c11a8b4bef61f2d71cf32b74a2d03c9c164a314ab332"}
Jul 06 01:20:47.896002 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.895636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt" event={"ID":"1018a839-0ecd-4609-900f-396112d157a1","Type":"ContainerStarted","Data":"da6068b0234fca3e1513ecdd671829414fda541068cd641a5726f841abf0ddaf"}
Jul 06 01:20:47.897803 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.897776    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff" event={"ID":"4f9f8db7-d5b6-456c-8f37-7237ff3384ab","Type":"ContainerStarted","Data":"3e6f7bfc57d58e364c420611e002c228ec473df2e80d1cdfa4c32e2523c0dc8f"}
Jul 06 01:20:47.897950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.897909    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:47.899535 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.899514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q" event={"ID":"776883bd-0078-48bc-adc7-06d04c2d0921","Type":"ContainerStarted","Data":"aa116fa8074e318db2c6b1316c89496abf753fca64d5179118b358c09c48ba09"}
Jul 06 01:20:47.900354 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.900298    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff"
Jul 06 01:20:47.902936 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.902917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerStarted","Data":"652c6d169be7ed5f5c63181007a151d13b7d328701d24fc11ff5c7b7c5c32785"}
Jul 06 01:20:47.903031 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.902944    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerStarted","Data":"4a190f62d27b749f31715937939d480172fba4201d9d38211b3eb212c6dfd57f"}
Jul 06 01:20:47.907276 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.907238    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ctfzm" podStartSLOduration=2.102494586 podStartE2EDuration="12.90722373s" totalImagesPullingTime="10.804729144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:47.906273856 +0000 UTC m=+47.751423266" watchObservedRunningTime="2026-07-06 01:20:47.90722373 +0000 UTC m=+47.752373148"
Jul 06 01:20:47.938312 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.938256    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7qs55" podStartSLOduration=35.099450913 podStartE2EDuration="45.938235099s" totalImagesPullingTime="10.838784186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:47.920530186 +0000 UTC m=+47.765679618" watchObservedRunningTime="2026-07-06 01:20:47.938235099 +0000 UTC m=+47.783384512"
Jul 06 01:20:47.938455 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.938363    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b5897d64b-t625q" podStartSLOduration=9.126519386 podStartE2EDuration="19.93835472s" totalImagesPullingTime="10.811835334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:47.936679939 +0000 UTC m=+47.781829350" watchObservedRunningTime="2026-07-06 01:20:47.93835472 +0000 UTC m=+47.783504136"
Jul 06 01:20:47.954032 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.953983    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt" podStartSLOduration=36.148733678 podStartE2EDuration="46.953961852s" totalImagesPullingTime="10.805228174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:47.953259365 +0000 UTC m=+47.798408776" watchObservedRunningTime="2026-07-06 01:20:47.953961852 +0000 UTC m=+47.799111266"
Jul 06 01:20:47.970814 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:47.970764    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6457ff646d-vxkff" podStartSLOduration=9.131977118 podStartE2EDuration="19.970745463s" totalImagesPullingTime="10.838768345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:47.970384184 +0000 UTC m=+47.815533599" watchObservedRunningTime="2026-07-06 01:20:47.970745463 +0000 UTC m=+47.815894872"
Jul 06 01:20:48.908718 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:48.908681    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerStarted","Data":"f2549070db50297b1151b70b0eb3d99eab5a3cc4949892e283be606b8ab0affa"}
Jul 06 01:20:48.928242 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:48.928199    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" podStartSLOduration=10.123645376 podStartE2EDuration="20.928184464s" totalImagesPullingTime="10.804539088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:48.926989386 +0000 UTC m=+48.772138801" watchObservedRunningTime="2026-07-06 01:20:48.928184464 +0000 UTC m=+48.773333876"
Jul 06 01:20:51.353029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:51.352991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:51.353053    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.353191    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.353265    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert podName:5d8e6636-d0a2-4a64-9f85-71c81dc678e4 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:07.353247532 +0000 UTC m=+67.198396925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-lkvb8" (UID: "5d8e6636-d0a2-4a64-9f85-71c81dc678e4") : secret "networking-console-plugin-cert" not found
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.353191    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.353298    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597fdbc7c4-qr7lc: secret "image-registry-tls" not found
Jul 06 01:20:51.353487 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.353363    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls podName:84cd28cc-a715-4e70-89e3-4e175fda4021 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:07.353344767 +0000 UTC m=+67.198494168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls") pod "image-registry-597fdbc7c4-qr7lc" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021") : secret "image-registry-tls" not found
Jul 06 01:20:51.453542 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:20:51.453514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:20:51.453691 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.453619    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 01:20:51.453691 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:20:51.453665    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls podName:1c306948-74cd-4b86-9383-c2687d0503c5 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:07.453652471 +0000 UTC m=+67.298801861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls") pod "dns-default-w6cs8" (UID: "1c306948-74cd-4b86-9383-c2687d0503c5") : secret "dns-default-metrics-tls" not found
Jul 06 01:21:00.799864 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:00.799838    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgsgd"
Jul 06 01:21:06.467165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.467103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:21:06.467575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.467193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:21:06.467575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.467244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:21:06.469462 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.469437    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 01:21:06.470221 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.470201    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 01:21:06.471618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.471598    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8vvx\" (UniqueName: \"kubernetes.io/projected/54e15649-83ed-4352-8a50-349ee45263eb-kube-api-access-h8vvx\") pod \"network-check-target-xlnzs\" (UID: \"54e15649-83ed-4352-8a50-349ee45263eb\") " pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:21:06.480067 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.480044    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc9f526-fc74-4399-95db-9cd060968c31-metrics-certs\") pod \"network-metrics-daemon-ktvvs\" (UID: \"7bc9f526-fc74-4399-95db-9cd060968c31\") " pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:21:06.480210 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.480195    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c277b2c-503f-48a2-9429-6ad2c2094008-original-pull-secret\") pod \"global-pull-secret-syncer-gc54p\" (UID: \"0c277b2c-503f-48a2-9429-6ad2c2094008\") " pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:21:06.565165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.565108    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kdr7\""
Jul 06 01:21:06.572811 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.572791    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:21:06.663288 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.663264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bqkt\""
Jul 06 01:21:06.666220 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.666202    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gc54p"
Jul 06 01:21:06.671249 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.671223    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ktvvs"
Jul 06 01:21:06.701645 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.701604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlnzs"]
Jul 06 01:21:06.704462 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:06.704436    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e15649_83ed_4352_8a50_349ee45263eb.slice/crio-d8a31d1c5855e1cce803168572950f6f26906063679622b6559fea818e379331 WatchSource:0}: Error finding container d8a31d1c5855e1cce803168572950f6f26906063679622b6559fea818e379331: Status 404 returned error can't find the container with id d8a31d1c5855e1cce803168572950f6f26906063679622b6559fea818e379331
Jul 06 01:21:06.804611 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.804583    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gc54p"]
Jul 06 01:21:06.807731 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:06.807707    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c277b2c_503f_48a2_9429_6ad2c2094008.slice/crio-32c7b86beb83257b2b5c309b0e0c3473e62d9227a9fb4eb1db63dffbbb81036c WatchSource:0}: Error finding container 32c7b86beb83257b2b5c309b0e0c3473e62d9227a9fb4eb1db63dffbbb81036c: Status 404 returned error can't find the container with id 32c7b86beb83257b2b5c309b0e0c3473e62d9227a9fb4eb1db63dffbbb81036c
Jul 06 01:21:06.816066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.815965    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ktvvs"]
Jul 06 01:21:06.818309 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:06.818285    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc9f526_fc74_4399_95db_9cd060968c31.slice/crio-07c0a2868ff831b3a85a34effb22a37c9b625e809948a5f560a24f8e50742b90 WatchSource:0}: Error finding container 07c0a2868ff831b3a85a34effb22a37c9b625e809948a5f560a24f8e50742b90: Status 404 returned error can't find the container with id 07c0a2868ff831b3a85a34effb22a37c9b625e809948a5f560a24f8e50742b90
Jul 06 01:21:06.965669 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.965632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlnzs" event={"ID":"54e15649-83ed-4352-8a50-349ee45263eb","Type":"ContainerStarted","Data":"478e52f730523a6a2aa75272d19ffe6a7f96990fe964c6cb99fad73f39f52ae6"}
Jul 06 01:21:06.965817 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.965690    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlnzs" event={"ID":"54e15649-83ed-4352-8a50-349ee45263eb","Type":"ContainerStarted","Data":"d8a31d1c5855e1cce803168572950f6f26906063679622b6559fea818e379331"}
Jul 06 01:21:06.965817 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.965718    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:21:06.966811 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.966789    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ktvvs" event={"ID":"7bc9f526-fc74-4399-95db-9cd060968c31","Type":"ContainerStarted","Data":"07c0a2868ff831b3a85a34effb22a37c9b625e809948a5f560a24f8e50742b90"}
Jul 06 01:21:06.967844 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.967827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gc54p" event={"ID":"0c277b2c-503f-48a2-9429-6ad2c2094008","Type":"ContainerStarted","Data":"32c7b86beb83257b2b5c309b0e0c3473e62d9227a9fb4eb1db63dffbbb81036c"}
Jul 06 01:21:06.981972 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:06.981903    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xlnzs" podStartSLOduration=65.981889408 podStartE2EDuration="1m5.981889408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:06.980784631 +0000 UTC m=+66.825934055" watchObservedRunningTime="2026-07-06 01:21:06.981889408 +0000 UTC m=+66.827038798"
Jul 06 01:21:07.376794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.376763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:07.376963 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.376810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:21:07.379307 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.379283    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"image-registry-597fdbc7c4-qr7lc\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") " pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:07.379400 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.379339    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d8e6636-d0a2-4a64-9f85-71c81dc678e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-lkvb8\" (UID: \"5d8e6636-d0a2-4a64-9f85-71c81dc678e4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:21:07.477733 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.477703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:21:07.481370 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.481343    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c306948-74cd-4b86-9383-c2687d0503c5-metrics-tls\") pod \"dns-default-w6cs8\" (UID: \"1c306948-74cd-4b86-9383-c2687d0503c5\") " pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:21:07.621250 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.621212    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qv997\""
Jul 06 01:21:07.629323 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.629250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:07.637670 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.637646    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hnpvc\""
Jul 06 01:21:07.646496 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.646474    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"
Jul 06 01:21:07.747501 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.745217    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g4lpx\""
Jul 06 01:21:07.753708 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.753489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:21:07.802968 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.802936    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:21:07.813705 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:07.813675    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cd28cc_a715_4e70_89e3_4e175fda4021.slice/crio-5cc049d29361bb70aad536d8c626792dfb87c4ace380ea403087ab8493f2ba1f WatchSource:0}: Error finding container 5cc049d29361bb70aad536d8c626792dfb87c4ace380ea403087ab8493f2ba1f: Status 404 returned error can't find the container with id 5cc049d29361bb70aad536d8c626792dfb87c4ace380ea403087ab8493f2ba1f
Jul 06 01:21:07.828399 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.828375    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8"]
Jul 06 01:21:07.831748 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:07.831722    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d8e6636_d0a2_4a64_9f85_71c81dc678e4.slice/crio-c89cf646c10a15357d8baec7bab1682a00c8ccd9d68fc4ca3e03afe173752b58 WatchSource:0}: Error finding container c89cf646c10a15357d8baec7bab1682a00c8ccd9d68fc4ca3e03afe173752b58: Status 404 returned error can't find the container with id c89cf646c10a15357d8baec7bab1682a00c8ccd9d68fc4ca3e03afe173752b58
Jul 06 01:21:07.930651 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.930620    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6cs8"]
Jul 06 01:21:07.934691 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:07.934663    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c306948_74cd_4b86_9383_c2687d0503c5.slice/crio-e4ae3457783e072a5c7bd5a16d596644b24720ba0510953d3d91244f900766d4 WatchSource:0}: Error finding container e4ae3457783e072a5c7bd5a16d596644b24720ba0510953d3d91244f900766d4: Status 404 returned error can't find the container with id e4ae3457783e072a5c7bd5a16d596644b24720ba0510953d3d91244f900766d4
Jul 06 01:21:07.972992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.972964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8" event={"ID":"5d8e6636-d0a2-4a64-9f85-71c81dc678e4","Type":"ContainerStarted","Data":"c89cf646c10a15357d8baec7bab1682a00c8ccd9d68fc4ca3e03afe173752b58"}
Jul 06 01:21:07.975639 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.975006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" event={"ID":"84cd28cc-a715-4e70-89e3-4e175fda4021","Type":"ContainerStarted","Data":"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"}
Jul 06 01:21:07.975639 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.975053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" event={"ID":"84cd28cc-a715-4e70-89e3-4e175fda4021","Type":"ContainerStarted","Data":"5cc049d29361bb70aad536d8c626792dfb87c4ace380ea403087ab8493f2ba1f"}
Jul 06 01:21:07.975639 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.975103    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:07.976582 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.976528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cs8" event={"ID":"1c306948-74cd-4b86-9383-c2687d0503c5","Type":"ContainerStarted","Data":"e4ae3457783e072a5c7bd5a16d596644b24720ba0510953d3d91244f900766d4"}
Jul 06 01:21:07.995673 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:07.995628    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" podStartSLOduration=65.995612716 podStartE2EDuration="1m5.995612716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:07.995053071 +0000 UTC m=+67.840202509" watchObservedRunningTime="2026-07-06 01:21:07.995612716 +0000 UTC m=+67.840762129"
Jul 06 01:21:08.984202 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:08.984169    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ktvvs" event={"ID":"7bc9f526-fc74-4399-95db-9cd060968c31","Type":"ContainerStarted","Data":"930af72e2270762864b56581d7a640b0c8e5dc2b13853ed7614773440ecf0269"}
Jul 06 01:21:08.984629 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:08.984226    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ktvvs" event={"ID":"7bc9f526-fc74-4399-95db-9cd060968c31","Type":"ContainerStarted","Data":"e0edc29a444541a6639d4482e23c6633ea9cf06ae9288b8b638713eaa211051a"}
Jul 06 01:21:09.000501 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:08.999346    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ktvvs" podStartSLOduration=66.182836039 podStartE2EDuration="1m7.999324903s" totalImagesPullingTime="1.816488864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:08.999063559 +0000 UTC m=+68.844213000" watchObservedRunningTime="2026-07-06 01:21:08.999324903 +0000 UTC m=+68.844474317"
Jul 06 01:21:11.999292 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:11.999254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cs8" event={"ID":"1c306948-74cd-4b86-9383-c2687d0503c5","Type":"ContainerStarted","Data":"e74e001da312544de43e67da52ad60f2fc912270707f7b5825d1b7bce5e51e90"}
Jul 06 01:21:11.999743 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:11.999315    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cs8" event={"ID":"1c306948-74cd-4b86-9383-c2687d0503c5","Type":"ContainerStarted","Data":"9ba585092efd5bfae6009f72a6c727a9605fe982ffc98fc1c009c05b9e2ab29f"}
Jul 06 01:21:11.999743 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:11.999401    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:21:12.000992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:12.000968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gc54p" event={"ID":"0c277b2c-503f-48a2-9429-6ad2c2094008","Type":"ContainerStarted","Data":"b91f5d4002e338a24648454bf87425c58af67ffda29feadbd4f7a0b7cb55d9e3"}
Jul 06 01:21:12.002380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:12.002352    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8" event={"ID":"5d8e6636-d0a2-4a64-9f85-71c81dc678e4","Type":"ContainerStarted","Data":"fe3c4fec767023fe381c052db8424ba7c6952faf10818e71de291c30204cb342"}
Jul 06 01:21:12.016813 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:12.016773    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w6cs8" podStartSLOduration=33.411337001 podStartE2EDuration="37.01675934s" totalImagesPullingTime="3.605422339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:12.015040163 +0000 UTC m=+71.860189593" watchObservedRunningTime="2026-07-06 01:21:12.01675934 +0000 UTC m=+71.861908751"
Jul 06 01:21:12.030272 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:12.030234    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gc54p" podStartSLOduration=66.284241388 podStartE2EDuration="1m11.030218378s" totalImagesPullingTime="4.74597699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:12.029519042 +0000 UTC m=+71.874668478" watchObservedRunningTime="2026-07-06 01:21:12.030218378 +0000 UTC m=+71.875367792"
Jul 06 01:21:12.044943 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:12.044904    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-lkvb8" podStartSLOduration=39.338519269 podStartE2EDuration="43.044864546s" totalImagesPullingTime="3.706345277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:12.044025625 +0000 UTC m=+71.889175053" watchObservedRunningTime="2026-07-06 01:21:12.044864546 +0000 UTC m=+71.890013957"
Jul 06 01:21:13.522094 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.522065    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:21:13.616856 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.616825    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-878457b9b-mn748"]
Jul 06 01:21:13.620853 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.620830    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.654207 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.654183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-878457b9b-mn748"]
Jul 06 01:21:13.723552 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-image-registry-private-configuration\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723554    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-installation-pull-secrets\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-bound-sa-token\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4xbc\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-kube-api-access-p4xbc\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723691    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d16168d-dc9a-46cb-86ff-006409ed781b-ca-trust-extracted\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723827 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-trusted-ca\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723827 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-certificates\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.723827 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.723815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-tls\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-certificates\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-tls\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-image-registry-private-configuration\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-installation-pull-secrets\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-bound-sa-token\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4xbc\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-kube-api-access-p4xbc\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824267    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d16168d-dc9a-46cb-86ff-006409ed781b-ca-trust-extracted\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-trusted-ca\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.824816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.824792    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d16168d-dc9a-46cb-86ff-006409ed781b-ca-trust-extracted\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.825277 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.825253    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-certificates\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.825664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.825645    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d16168d-dc9a-46cb-86ff-006409ed781b-trusted-ca\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.826842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.826814    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-installation-pull-secrets\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.826925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.826821    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-registry-tls\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.826925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.826874    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d16168d-dc9a-46cb-86ff-006409ed781b-image-registry-private-configuration\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.836888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.836865    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-bound-sa-token\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.838046 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.838023    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4xbc\" (UniqueName: \"kubernetes.io/projected/8d16168d-dc9a-46cb-86ff-006409ed781b-kube-api-access-p4xbc\") pod \"image-registry-878457b9b-mn748\" (UID: \"8d16168d-dc9a-46cb-86ff-006409ed781b\") " pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:13.929857 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:13.929831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:14.061513 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:14.061479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-878457b9b-mn748"]
Jul 06 01:21:14.064464 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:14.064435    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d16168d_dc9a_46cb_86ff_006409ed781b.slice/crio-03e211fa092dbb2424d95937a20099b2b0933ac16db181be3e52178e868c766d WatchSource:0}: Error finding container 03e211fa092dbb2424d95937a20099b2b0933ac16db181be3e52178e868c766d: Status 404 returned error can't find the container with id 03e211fa092dbb2424d95937a20099b2b0933ac16db181be3e52178e868c766d
Jul 06 01:21:15.013438 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:15.013405    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-878457b9b-mn748" event={"ID":"8d16168d-dc9a-46cb-86ff-006409ed781b","Type":"ContainerStarted","Data":"77a36ecd0080e8f179d0717647bdd039861ad99e1f70e883f737c9cecf634c86"}
Jul 06 01:21:15.013815 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:15.013450    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-878457b9b-mn748" event={"ID":"8d16168d-dc9a-46cb-86ff-006409ed781b","Type":"ContainerStarted","Data":"03e211fa092dbb2424d95937a20099b2b0933ac16db181be3e52178e868c766d"}
Jul 06 01:21:15.013815 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:15.013531    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:15.036423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:15.036378    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-878457b9b-mn748" podStartSLOduration=2.036362381 podStartE2EDuration="2.036362381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:15.035252504 +0000 UTC m=+74.880401938" watchObservedRunningTime="2026-07-06 01:21:15.036362381 +0000 UTC m=+74.881511793"
Jul 06 01:21:20.975502 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.975468    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-twncn"]
Jul 06 01:21:20.990543 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.990520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:20.996688 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.996668    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 01:21:20.998223 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.998199    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 01:21:20.998531 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.998506    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxc9g\""
Jul 06 01:21:20.998644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:20.998575    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 01:21:21.000290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.000276    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 01:21:21.016406 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.016380    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-twncn"]
Jul 06 01:21:21.077194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.077165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992bbf59-52ef-431f-b678-552c6ef05c3b-data-volume\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.077402 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.077230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.077402 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.077281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.077402 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.077321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9nh\" (UniqueName: \"kubernetes.io/projected/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-api-access-sl9nh\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.077402 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.077395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992bbf59-52ef-431f-b678-552c6ef05c3b-crio-socket\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178433 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992bbf59-52ef-431f-b678-552c6ef05c3b-crio-socket\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178598 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178483    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992bbf59-52ef-431f-b678-552c6ef05c3b-crio-socket\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178598 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992bbf59-52ef-431f-b678-552c6ef05c3b-data-volume\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178598 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178598 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9nh\" (UniqueName: \"kubernetes.io/projected/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-api-access-sl9nh\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.178794 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.178652    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:21.178794 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.178706    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 01:21:21.178794 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.178713    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:21.678697815 +0000 UTC m=+81.523847218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:21.178794 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.178775    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:21.678759811 +0000 UTC m=+81.523909207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : configmap "kube-rbac-proxy" not found
Jul 06 01:21:21.179033 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.178828    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992bbf59-52ef-431f-b678-552c6ef05c3b-data-volume\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.186620 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.186601    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9nh\" (UniqueName: \"kubernetes.io/projected/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-api-access-sl9nh\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.682022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.681906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.682022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:21.681944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:21.682238 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.682025    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 01:21:21.682238 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.682056    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:21.682238 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.682078    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:22.682064 +0000 UTC m=+82.527213390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : configmap "kube-rbac-proxy" not found
Jul 06 01:21:21.682238 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:21.682106    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:22.682094038 +0000 UTC m=+82.527243427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:22.007559 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:22.007472    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w6cs8"
Jul 06 01:21:22.396031 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:22.396004    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cs8_1c306948-74cd-4b86-9383-c2687d0503c5/dns/0.log"
Jul 06 01:21:22.567286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:22.567259    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cs8_1c306948-74cd-4b86-9383-c2687d0503c5/kube-rbac-proxy/0.log"
Jul 06 01:21:22.689196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:22.689101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:22.689196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:22.689155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:22.689363 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:22.689215    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 01:21:22.689363 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:22.689241    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:22.689363 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:22.689263    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:24.689250019 +0000 UTC m=+84.534399409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : configmap "kube-rbac-proxy" not found
Jul 06 01:21:22.689363 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:22.689294    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:24.689280819 +0000 UTC m=+84.534430208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:23.168143 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:23.168099    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s6r8x_e0122480-e48e-449b-b4eb-2f78298552b4/dns-node-resolver/0.log"
Jul 06 01:21:23.366528 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:23.366498    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-597fdbc7c4-qr7lc_84cd28cc-a715-4e70-89e3-4e175fda4021/registry/0.log"
Jul 06 01:21:23.527564 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:23.527483    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:23.769049 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:23.769016    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-878457b9b-mn748_8d16168d-dc9a-46cb-86ff-006409ed781b/registry/0.log"
Jul 06 01:21:23.969038 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:23.969011    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26d9w_1d794170-f755-4376-842b-17583c87c9c5/node-ca/0.log"
Jul 06 01:21:24.707353 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:24.707317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:24.707814 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:24.707361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:24.707814 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:24.707458    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 01:21:24.707814 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:24.707519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:28.707497506 +0000 UTC m=+88.552646899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : configmap "kube-rbac-proxy" not found
Jul 06 01:21:24.707814 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:24.707526    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:24.707814 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:24.707660    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls podName:992bbf59-52ef-431f-b678-552c6ef05c3b nodeName:}" failed. No retries permitted until 2026-07-06 01:21:28.707639731 +0000 UTC m=+88.552789137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-twncn" (UID: "992bbf59-52ef-431f-b678-552c6ef05c3b") : secret "insights-runtime-extractor-tls" not found
Jul 06 01:21:24.966022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:24.965949    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ctfzm_6c131c57-4eb6-45fd-9dd8-76c09f647891/serve-healthcheck-canary/0.log"
Jul 06 01:21:28.735410 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.735377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:28.735870 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.735412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:28.735943 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.735926    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992bbf59-52ef-431f-b678-552c6ef05c3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:28.737891 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.737870    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992bbf59-52ef-431f-b678-552c6ef05c3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-twncn\" (UID: \"992bbf59-52ef-431f-b678-552c6ef05c3b\") " pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:28.799546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.799519    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-twncn"
Jul 06 01:21:28.951818 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:28.951784    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-twncn"]
Jul 06 01:21:28.956344 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:28.956319    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992bbf59_52ef_431f_b678_552c6ef05c3b.slice/crio-0c9517472b92c3c5939b9b6b480622dfc8273d46b3987e1b41d05d40431bfa92 WatchSource:0}: Error finding container 0c9517472b92c3c5939b9b6b480622dfc8273d46b3987e1b41d05d40431bfa92: Status 404 returned error can't find the container with id 0c9517472b92c3c5939b9b6b480622dfc8273d46b3987e1b41d05d40431bfa92
Jul 06 01:21:29.060770 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.060686    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-twncn" event={"ID":"992bbf59-52ef-431f-b678-552c6ef05c3b","Type":"ContainerStarted","Data":"0c9517472b92c3c5939b9b6b480622dfc8273d46b3987e1b41d05d40431bfa92"}
Jul 06 01:21:29.796541 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.796194    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n52xs"]
Jul 06 01:21:29.800103 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.800081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.804489 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.803528    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 01:21:29.804489 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.803730    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 01:21:29.804489 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.803905    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 01:21:29.804489 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.804074    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q8hgd\""
Jul 06 01:21:29.804489 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.804297    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 01:21:29.804869 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.804573    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 01:21:29.804869 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.804836    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945777    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-metrics-client-ca\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-root\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-wtmp\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.945906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-sys\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.946403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-textfile\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.946403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.945985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p95xb\" (UniqueName: \"kubernetes.io/projected/97ad9e35-0895-466d-bfe3-2ba723e75bad-kube-api-access-p95xb\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:29.946403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:29.946062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-accelerators-collector-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.046950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.046866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-accelerators-collector-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.046950 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.046920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.046952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-metrics-client-ca\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-root\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-wtmp\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:30.047099    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047148    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-sys\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-textfile\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047209 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:30.047205    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls podName:97ad9e35-0895-466d-bfe3-2ba723e75bad nodeName:}" failed. No retries permitted until 2026-07-06 01:21:30.547182915 +0000 UTC m=+90.392332308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls") pod "node-exporter-n52xs" (UID: "97ad9e35-0895-466d-bfe3-2ba723e75bad") : secret "node-exporter-tls" not found
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047202    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-root\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p95xb\" (UniqueName: \"kubernetes.io/projected/97ad9e35-0895-466d-bfe3-2ba723e75bad-kube-api-access-p95xb\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047356    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-wtmp\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047405    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ad9e35-0895-466d-bfe3-2ba723e75bad-sys\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047477    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-textfile\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047575    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-accelerators-collector-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.047925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.047909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97ad9e35-0895-466d-bfe3-2ba723e75bad-metrics-client-ca\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.049940 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.049903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.059818 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.059799    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p95xb\" (UniqueName: \"kubernetes.io/projected/97ad9e35-0895-466d-bfe3-2ba723e75bad-kube-api-access-p95xb\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.551418 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.551381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.554070 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.554044    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97ad9e35-0895-466d-bfe3-2ba723e75bad-node-exporter-tls\") pod \"node-exporter-n52xs\" (UID: \"97ad9e35-0895-466d-bfe3-2ba723e75bad\") " pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:30.715207 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:30.715164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n52xs"
Jul 06 01:21:32.615369 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:21:32.615332    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ad9e35_0895_466d_bfe3_2ba723e75bad.slice/crio-20fc671b3ea6dae395bba453bf3c7570d08c6ed8557336774a4c58b8ad4f3c8c WatchSource:0}: Error finding container 20fc671b3ea6dae395bba453bf3c7570d08c6ed8557336774a4c58b8ad4f3c8c: Status 404 returned error can't find the container with id 20fc671b3ea6dae395bba453bf3c7570d08c6ed8557336774a4c58b8ad4f3c8c
Jul 06 01:21:33.079140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:33.079076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-twncn" event={"ID":"992bbf59-52ef-431f-b678-552c6ef05c3b","Type":"ContainerStarted","Data":"55b2a991ed771d308a356dd46f2b37013d7a4c19b99a8a9fce70ef41c2517a04"}
Jul 06 01:21:33.080762 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:33.080691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n52xs" event={"ID":"97ad9e35-0895-466d-bfe3-2ba723e75bad","Type":"ContainerStarted","Data":"20fc671b3ea6dae395bba453bf3c7570d08c6ed8557336774a4c58b8ad4f3c8c"}
Jul 06 01:21:34.087279 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:34.087239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-twncn" event={"ID":"992bbf59-52ef-431f-b678-552c6ef05c3b","Type":"ContainerStarted","Data":"b3c47337b221793e33cc42e3c9add4fc11d118ade9a53d002e675d9f330589c2"}
Jul 06 01:21:34.089077 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:34.089046    2567 generic.go:356] "Generic (PLEG): container finished" podID="97ad9e35-0895-466d-bfe3-2ba723e75bad" containerID="74bcdd612781eeaa3e8e3fee0fadd009d85abb8cf3bd209b3157a1d049eca879" exitCode=0
Jul 06 01:21:34.089212 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:34.089159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n52xs" event={"ID":"97ad9e35-0895-466d-bfe3-2ba723e75bad","Type":"ContainerDied","Data":"74bcdd612781eeaa3e8e3fee0fadd009d85abb8cf3bd209b3157a1d049eca879"}
Jul 06 01:21:35.095143 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:35.095091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n52xs" event={"ID":"97ad9e35-0895-466d-bfe3-2ba723e75bad","Type":"ContainerStarted","Data":"618b57a8ac079cac612049a2a32e06e1b48d5f73213f0a2643185ea538d9bb53"}
Jul 06 01:21:35.095609 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:35.095167    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n52xs" event={"ID":"97ad9e35-0895-466d-bfe3-2ba723e75bad","Type":"ContainerStarted","Data":"d5aeb62afcdb1f99ea93f17aae979f089160f72570ba1d6fd99d57539269548b"}
Jul 06 01:21:35.114135 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:35.114066    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n52xs" podStartSLOduration=5.194216778 podStartE2EDuration="6.114047072s" totalImagesPullingTime="848.377685ms" totalInitContainerRuntime="71.452609ms" isStatefulPod=true podCreationTimestamp="2026-07-06 01:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:35.111881618 +0000 UTC m=+94.957031054" watchObservedRunningTime="2026-07-06 01:21:35.114047072 +0000 UTC m=+94.959196486"
Jul 06 01:21:36.023268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:36.023241    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-878457b9b-mn748"
Jul 06 01:21:37.107175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:37.107144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-twncn" event={"ID":"992bbf59-52ef-431f-b678-552c6ef05c3b","Type":"ContainerStarted","Data":"895d04acdc0e0789b337983b8ef84799f97850dd36d987c9439d4fcac51cd70a"}
Jul 06 01:21:37.124338 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:37.124290    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-twncn" podStartSLOduration=10.169701521 podStartE2EDuration="17.12427327s" totalImagesPullingTime="6.954571749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:21:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:37.123245593 +0000 UTC m=+96.968395023" watchObservedRunningTime="2026-07-06 01:21:37.12427327 +0000 UTC m=+96.969422682"
Jul 06 01:21:37.979679 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:37.979648    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xlnzs"
Jul 06 01:21:38.541767 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.541720    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" podUID="84cd28cc-a715-4e70-89e3-4e175fda4021" containerName="registry" containerID="cri-o://d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2" gracePeriod=30
Jul 06 01:21:38.777789 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.777765    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:38.918264 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918235    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918280    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918297    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918511 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918414    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42rdx\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918511 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918456    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918511 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918491    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918660 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918518    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918660 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918553    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") pod \"84cd28cc-a715-4e70-89e3-4e175fda4021\" (UID: \"84cd28cc-a715-4e70-89e3-4e175fda4021\") "
Jul 06 01:21:38.918821 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918694    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:21:38.918946 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.918892    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:21:38.920935 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.920904    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:21:38.921153 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.920967    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx" (OuterVolumeSpecName: "kube-api-access-42rdx") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "kube-api-access-42rdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:21:38.921153 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.921069    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:21:38.921342 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.921321    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:21:38.921403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.921351    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:21:38.926794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:38.926770    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "84cd28cc-a715-4e70-89e3-4e175fda4021" (UID: "84cd28cc-a715-4e70-89e3-4e175fda4021"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:21:39.019707 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019682    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-trusted-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019707 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019704    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-image-registry-private-configuration\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019707 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019713    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019876 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019722    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84cd28cc-a715-4e70-89e3-4e175fda4021-ca-trust-extracted\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019876 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019730    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84cd28cc-a715-4e70-89e3-4e175fda4021-installation-pull-secrets\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019876 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019739    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84cd28cc-a715-4e70-89e3-4e175fda4021-registry-certificates\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019876 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019746    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42rdx\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-kube-api-access-42rdx\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.019876 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.019754    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84cd28cc-a715-4e70-89e3-4e175fda4021-bound-sa-token\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:21:39.115635 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.115606    2567 generic.go:356] "Generic (PLEG): container finished" podID="84cd28cc-a715-4e70-89e3-4e175fda4021" containerID="d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2" exitCode=0
Jul 06 01:21:39.115755 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.115651    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc"
Jul 06 01:21:39.115755 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.115695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" event={"ID":"84cd28cc-a715-4e70-89e3-4e175fda4021","Type":"ContainerDied","Data":"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"}
Jul 06 01:21:39.115755 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.115739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-597fdbc7c4-qr7lc" event={"ID":"84cd28cc-a715-4e70-89e3-4e175fda4021","Type":"ContainerDied","Data":"5cc049d29361bb70aad536d8c626792dfb87c4ace380ea403087ab8493f2ba1f"}
Jul 06 01:21:39.115877 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.115774    2567 scope.go:117] "RemoveContainer" containerID="d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"
Jul 06 01:21:39.124143 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.124099    2567 scope.go:117] "RemoveContainer" containerID="d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"
Jul 06 01:21:39.124424 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:21:39.124404    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2\": container with ID starting with d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2 not found: ID does not exist" containerID="d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"
Jul 06 01:21:39.124488 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.124432    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2"} err="failed to get container status \"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2\": rpc error: code = NotFound desc = could not find container \"d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2\": container with ID starting with d7d3c20d49902ecb107601d03e86f92c7f0623539b4da171d6fef95a286fd3c2 not found: ID does not exist"
Jul 06 01:21:39.136694 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.136671    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:21:39.140403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:39.140385    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-597fdbc7c4-qr7lc"]
Jul 06 01:21:40.654982 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:21:40.654952    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84cd28cc-a715-4e70-89e3-4e175fda4021" path="/var/lib/kubelet/pods/84cd28cc-a715-4e70-89e3-4e175fda4021/volumes"
Jul 06 01:22:14.233679 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:14.233645    2567 generic.go:356] "Generic (PLEG): container finished" podID="1018a839-0ecd-4609-900f-396112d157a1" containerID="da6068b0234fca3e1513ecdd671829414fda541068cd641a5726f841abf0ddaf" exitCode=0
Jul 06 01:22:14.234104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:14.233719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt" event={"ID":"1018a839-0ecd-4609-900f-396112d157a1","Type":"ContainerDied","Data":"da6068b0234fca3e1513ecdd671829414fda541068cd641a5726f841abf0ddaf"}
Jul 06 01:22:14.234104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:14.234012    2567 scope.go:117] "RemoveContainer" containerID="da6068b0234fca3e1513ecdd671829414fda541068cd641a5726f841abf0ddaf"
Jul 06 01:22:15.238488 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:15.238458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-z7rjt" event={"ID":"1018a839-0ecd-4609-900f-396112d157a1","Type":"ContainerStarted","Data":"be0e21ac53aee3479527d880b335f3d1bd525a2f6f22174a76de6cbbab2198b4"}
Jul 06 01:22:25.911384 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:25.911331    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" podUID="7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 01:22:35.911086 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:35.911045    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" podUID="7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 01:22:45.910807 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:45.910768    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" podUID="7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 01:22:45.911230 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:45.910843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn"
Jul 06 01:22:45.911322 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:45.911304    2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"f2549070db50297b1151b70b0eb3d99eab5a3cc4949892e283be606b8ab0affa"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 06 01:22:45.911357 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:45.911342    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" podUID="7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9" containerName="service-proxy" containerID="cri-o://f2549070db50297b1151b70b0eb3d99eab5a3cc4949892e283be606b8ab0affa" gracePeriod=30
Jul 06 01:22:46.341351 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:46.341263    2567 generic.go:356] "Generic (PLEG): container finished" podID="7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9" containerID="f2549070db50297b1151b70b0eb3d99eab5a3cc4949892e283be606b8ab0affa" exitCode=2
Jul 06 01:22:46.341481 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:46.341346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerDied","Data":"f2549070db50297b1151b70b0eb3d99eab5a3cc4949892e283be606b8ab0affa"}
Jul 06 01:22:46.341481 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:22:46.341396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68f748fcc6-rt7fn" event={"ID":"7ae3b90a-cc41-4e13-85c6-44c1dc8eb4c9","Type":"ContainerStarted","Data":"06508002cfae2a48ded577c95ad80217632fb5b174fac9648ea7bf949e6319ae"}
Jul 06 01:25:00.556375 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:25:00.556350    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:25:00.560054 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:25:00.560033    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 01:25:00.564348 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:25:00.564330    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:26:09.240799 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.240720    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"]
Jul 06 01:26:09.241204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.240946    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84cd28cc-a715-4e70-89e3-4e175fda4021" containerName="registry"
Jul 06 01:26:09.241204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.240956    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84cd28cc-a715-4e70-89e3-4e175fda4021" containerName="registry"
Jul 06 01:26:09.241204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.241009    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84cd28cc-a715-4e70-89e3-4e175fda4021" containerName="registry"
Jul 06 01:26:09.243930 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.243914    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.246722 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.246699    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 01:26:09.247302 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.247286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 01:26:09.247522 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.247500    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mxjmq\""
Jul 06 01:26:09.247671 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.247660    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 01:26:09.248731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.248714    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 01:26:09.270907 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.270889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"]
Jul 06 01:26:09.295910 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.295889    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.296012 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.295936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqb7\" (UniqueName: \"kubernetes.io/projected/ae8b3a91-62b7-46b5-928d-88b105a0b778-kube-api-access-jbqb7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.296012 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.295960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.296085 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.296030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.296140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.296084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqb7\" (UniqueName: \"kubernetes.io/projected/ae8b3a91-62b7-46b5-928d-88b105a0b778-kube-api-access-jbqb7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.397599 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.397579    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.399575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.399549    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8b3a91-62b7-46b5-928d-88b105a0b778-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.399575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.399567    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.399726 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.399709    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8b3a91-62b7-46b5-928d-88b105a0b778-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.408328 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.408303    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqb7\" (UniqueName: \"kubernetes.io/projected/ae8b3a91-62b7-46b5-928d-88b105a0b778-kube-api-access-jbqb7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-ccmkr\" (UID: \"ae8b3a91-62b7-46b5-928d-88b105a0b778\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.553871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.553794    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:09.693845 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.693816    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"]
Jul 06 01:26:09.697082 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:09.697058    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8b3a91_62b7_46b5_928d_88b105a0b778.slice/crio-d1452284871b09f4fea485bd6ccebf90acdd4cc557ade337e37bff4972adbdc1 WatchSource:0}: Error finding container d1452284871b09f4fea485bd6ccebf90acdd4cc557ade337e37bff4972adbdc1: Status 404 returned error can't find the container with id d1452284871b09f4fea485bd6ccebf90acdd4cc557ade337e37bff4972adbdc1
Jul 06 01:26:09.698717 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:09.698701    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 01:26:10.016804 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:10.016773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr" event={"ID":"ae8b3a91-62b7-46b5-928d-88b105a0b778","Type":"ContainerStarted","Data":"d1452284871b09f4fea485bd6ccebf90acdd4cc557ade337e37bff4972adbdc1"}
Jul 06 01:26:13.030801 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:13.030725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr" event={"ID":"ae8b3a91-62b7-46b5-928d-88b105a0b778","Type":"ContainerStarted","Data":"10ee6dbe16c070eca837340c9acd93228239165c36b0c35572154f36c74c8cb7"}
Jul 06 01:26:14.036272 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:14.036242    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae8b3a91-62b7-46b5-928d-88b105a0b778" containerID="10ee6dbe16c070eca837340c9acd93228239165c36b0c35572154f36c74c8cb7" exitCode=0
Jul 06 01:26:14.036658 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:14.036331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr" event={"ID":"ae8b3a91-62b7-46b5-928d-88b105a0b778","Type":"ContainerDied","Data":"10ee6dbe16c070eca837340c9acd93228239165c36b0c35572154f36c74c8cb7"}
Jul 06 01:26:15.043265 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:15.043225    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr" event={"ID":"ae8b3a91-62b7-46b5-928d-88b105a0b778","Type":"ContainerStarted","Data":"379eb43ded10709a6a0e626fb092bfaa069a7e05b5ad9755bc22c63b092ac64e"}
Jul 06 01:26:15.043658 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:15.043378    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:15.063731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:15.063687    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr" podStartSLOduration=1.665629763 podStartE2EDuration="6.063672131s" totalImagesPullingTime="3.370430915s" totalInitContainerRuntime="1.027611453s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:15.062507237 +0000 UTC m=+374.907656650" watchObservedRunningTime="2026-07-06 01:26:15.063672131 +0000 UTC m=+374.908821543"
Jul 06 01:26:26.049279 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:26.049246    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-ccmkr"
Jul 06 01:26:29.440050 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.440018    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"]
Jul 06 01:26:29.445629 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.445609    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.447959 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.447931    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 01:26:29.448768 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.448742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 01:26:29.448875 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.448774    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 01:26:29.448875 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.448780    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 01:26:29.448875 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.448774    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hdcln\""
Jul 06 01:26:29.456504 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.456486    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"]
Jul 06 01:26:29.546362 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.546323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd0b9b58-44f5-4432-ad90-2afc08769bed-tmp\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.546524 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.546377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd0b9b58-44f5-4432-ad90-2afc08769bed-tls-certs\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.546524 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.546395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7xtk\" (UniqueName: \"kubernetes.io/projected/dd0b9b58-44f5-4432-ad90-2afc08769bed-kube-api-access-l7xtk\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.588704 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.588669    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mqd8"]
Jul 06 01:26:29.591098 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.591081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.594368 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.594349    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2nm4v\""
Jul 06 01:26:29.594475 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.594381    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 01:26:29.606578 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.606515    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mqd8"]
Jul 06 01:26:29.647252 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.647215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd0b9b58-44f5-4432-ad90-2afc08769bed-tmp\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.647429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.647310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd0b9b58-44f5-4432-ad90-2afc08769bed-tls-certs\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.647429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.647337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7xtk\" (UniqueName: \"kubernetes.io/projected/dd0b9b58-44f5-4432-ad90-2afc08769bed-kube-api-access-l7xtk\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.649953 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.649921    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd0b9b58-44f5-4432-ad90-2afc08769bed-tls-certs\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.650077 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.649967    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd0b9b58-44f5-4432-ad90-2afc08769bed-tmp\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.655015 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.654986    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7xtk\" (UniqueName: \"kubernetes.io/projected/dd0b9b58-44f5-4432-ad90-2afc08769bed-kube-api-access-l7xtk\") pod \"kube-auth-proxy-799868c9c7-mzzh5\" (UID: \"dd0b9b58-44f5-4432-ad90-2afc08769bed\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.748562 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.748469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg28x\" (UniqueName: \"kubernetes.io/projected/37ce2ac4-c403-44f2-9760-3ceaacf3584d-kube-api-access-sg28x\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.748562 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.748527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.755448 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.755419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"
Jul 06 01:26:29.849677 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.849643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.849864 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.849698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg28x\" (UniqueName: \"kubernetes.io/projected/37ce2ac4-c403-44f2-9760-3ceaacf3584d-kube-api-access-sg28x\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.850038 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:26:29.850021    2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 01:26:29.850098 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:26:29.850072    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert podName:37ce2ac4-c403-44f2-9760-3ceaacf3584d nodeName:}" failed. No retries permitted until 2026-07-06 01:26:30.350058098 +0000 UTC m=+390.195207488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert") pod "odh-model-controller-7998c67dc4-2mqd8" (UID: "37ce2ac4-c403-44f2-9760-3ceaacf3584d") : secret "odh-model-controller-webhook-cert" not found
Jul 06 01:26:29.860319 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.860276    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg28x\" (UniqueName: \"kubernetes.io/projected/37ce2ac4-c403-44f2-9760-3ceaacf3584d-kube-api-access-sg28x\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:29.889449 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:29.889417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5"]
Jul 06 01:26:29.892381 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:29.892357    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0b9b58_44f5_4432_ad90_2afc08769bed.slice/crio-280120df7ba572b3092f09f16a6a1b2a3dc9603e62795d3178840db57cf89945 WatchSource:0}: Error finding container 280120df7ba572b3092f09f16a6a1b2a3dc9603e62795d3178840db57cf89945: Status 404 returned error can't find the container with id 280120df7ba572b3092f09f16a6a1b2a3dc9603e62795d3178840db57cf89945
Jul 06 01:26:30.094642 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.094563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5" event={"ID":"dd0b9b58-44f5-4432-ad90-2afc08769bed","Type":"ContainerStarted","Data":"280120df7ba572b3092f09f16a6a1b2a3dc9603e62795d3178840db57cf89945"}
Jul 06 01:26:30.260946 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.260912    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pd7kl"]
Jul 06 01:26:30.263711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.263691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.267701 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.267348    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 01:26:30.267701 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.267570    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-67hgw\""
Jul 06 01:26:30.277282 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.277257    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pd7kl"]
Jul 06 01:26:30.353888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.353801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7982eb9-e361-47df-ba5f-4ab83bb12257-tls-certs\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.353888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.353846    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtgf\" (UniqueName: \"kubernetes.io/projected/b7982eb9-e361-47df-ba5f-4ab83bb12257-kube-api-access-hxtgf\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.354077 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.353886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:30.356497 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.356470    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ce2ac4-c403-44f2-9760-3ceaacf3584d-cert\") pod \"odh-model-controller-7998c67dc4-2mqd8\" (UID: \"37ce2ac4-c403-44f2-9760-3ceaacf3584d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:30.454695 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.454664    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7982eb9-e361-47df-ba5f-4ab83bb12257-tls-certs\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.454695 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.454703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtgf\" (UniqueName: \"kubernetes.io/projected/b7982eb9-e361-47df-ba5f-4ab83bb12257-kube-api-access-hxtgf\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.457533 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.457510    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7982eb9-e361-47df-ba5f-4ab83bb12257-tls-certs\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.461813 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.461795    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtgf\" (UniqueName: \"kubernetes.io/projected/b7982eb9-e361-47df-ba5f-4ab83bb12257-kube-api-access-hxtgf\") pod \"model-serving-api-54d89bd5fc-pd7kl\" (UID: \"b7982eb9-e361-47df-ba5f-4ab83bb12257\") " pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.500049 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.500028    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:30.576225 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.576194    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:30.680594 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.680545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mqd8"]
Jul 06 01:26:30.680681 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.680602    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:30.686152 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.684536    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.686383 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.686364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:30.687341 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.686843    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 01:26:30.687341 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.686868    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 01:26:30.687341 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.687045    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gq4dv\""
Jul 06 01:26:30.687341 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.687081    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 01:26:30.687341 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.687254    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 01:26:30.753486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.753457    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pd7kl"]
Jul 06 01:26:30.756348 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:30.756322    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7982eb9_e361_47df_ba5f_4ab83bb12257.slice/crio-c0f3b06234a2c4e3e7eb5cd2e31344852951b6aea14b5c43782a528c39f19fba WatchSource:0}: Error finding container c0f3b06234a2c4e3e7eb5cd2e31344852951b6aea14b5c43782a528c39f19fba: Status 404 returned error can't find the container with id c0f3b06234a2c4e3e7eb5cd2e31344852951b6aea14b5c43782a528c39f19fba
Jul 06 01:26:30.866472 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69gtc\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866513    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866809 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866809 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.866809 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.866746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.967784 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.967784 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69gtc\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.967784 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967750    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.967784 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.968087 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.968087 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.968087 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.967868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.968790 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.968573    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.970535 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.970509    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.970634 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.970599    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.970688 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.970669    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.970939 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.970905    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.977029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.977010    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69gtc\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.977379 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.977353    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token\") pod \"istiod-openshift-gateway-7574459656-chtsq\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:30.998423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:30.998403    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:31.106321 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:31.106279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8" event={"ID":"37ce2ac4-c403-44f2-9760-3ceaacf3584d","Type":"ContainerStarted","Data":"6564464cd1bee4c2abba0419924c0718d51075d1d2b761c9f81d4c52443b2d4c"}
Jul 06 01:26:31.107847 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:31.107818    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl" event={"ID":"b7982eb9-e361-47df-ba5f-4ab83bb12257","Type":"ContainerStarted","Data":"c0f3b06234a2c4e3e7eb5cd2e31344852951b6aea14b5c43782a528c39f19fba"}
Jul 06 01:26:31.149852 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:31.149779    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:31.155823 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:31.155794    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb05abb2_c979_4740_aca5_7b5e835cb617.slice/crio-c9cf8124dffbeecf017667d0567e9b054c82e01ec0e7c316f8840f95509aba47 WatchSource:0}: Error finding container c9cf8124dffbeecf017667d0567e9b054c82e01ec0e7c316f8840f95509aba47: Status 404 returned error can't find the container with id c9cf8124dffbeecf017667d0567e9b054c82e01ec0e7c316f8840f95509aba47
Jul 06 01:26:32.119215 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:32.119173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" event={"ID":"bb05abb2-c979-4740-aca5-7b5e835cb617","Type":"ContainerStarted","Data":"c9cf8124dffbeecf017667d0567e9b054c82e01ec0e7c316f8840f95509aba47"}
Jul 06 01:26:35.779215 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:35.779174    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 06 01:26:35.779622 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:35.779256    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 06 01:26:36.144339 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.144294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5" event={"ID":"dd0b9b58-44f5-4432-ad90-2afc08769bed","Type":"ContainerStarted","Data":"f324dfcec9f199b945a197debdc76bd934bd95e19d9375f9f37ab809354d12dd"}
Jul 06 01:26:36.146290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.146265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl" event={"ID":"b7982eb9-e361-47df-ba5f-4ab83bb12257","Type":"ContainerStarted","Data":"170eb5bbaeadd6c79c3547b5b8061014271ee5683a61ad925b969f54f3969e17"}
Jul 06 01:26:36.146490 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.146430    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:26:36.148661 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.148637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" event={"ID":"bb05abb2-c979-4740-aca5-7b5e835cb617","Type":"ContainerStarted","Data":"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"}
Jul 06 01:26:36.148811 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.148792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:36.150799 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.150741    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-chtsq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 01:26:36.150949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.150797    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 01:26:36.150949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.150899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8" event={"ID":"37ce2ac4-c403-44f2-9760-3ceaacf3584d","Type":"ContainerStarted","Data":"b9259cbf048f87c3390723ef8efa0d7e84ddc5a7350984c49303d07cf5bc81f9"}
Jul 06 01:26:36.151217 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.151065    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:36.164000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.163947    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-799868c9c7-mzzh5" podStartSLOduration=1.349587313 podStartE2EDuration="7.163929916s" totalImagesPullingTime="5.814342603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:36.162342844 +0000 UTC m=+396.007492284" watchObservedRunningTime="2026-07-06 01:26:36.163929916 +0000 UTC m=+396.009079329"
Jul 06 01:26:36.178529 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.178476    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8" podStartSLOduration=2.073051042 podStartE2EDuration="7.178457799s" totalImagesPullingTime="5.105406757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:36.177350786 +0000 UTC m=+396.022500223" watchObservedRunningTime="2026-07-06 01:26:36.178457799 +0000 UTC m=+396.023607213"
Jul 06 01:26:36.193273 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.193222    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl" podStartSLOduration=1.1813248490000001 podStartE2EDuration="6.193204638s" totalImagesPullingTime="5.011879789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:36.191470982 +0000 UTC m=+396.036620392" watchObservedRunningTime="2026-07-06 01:26:36.193204638 +0000 UTC m=+396.038354051"
Jul 06 01:26:36.210670 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:36.210619    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" podStartSLOduration=1.590148189 podStartE2EDuration="6.210601681s" totalImagesPullingTime="4.620453492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:36.209331814 +0000 UTC m=+396.054481230" watchObservedRunningTime="2026-07-06 01:26:36.210601681 +0000 UTC m=+396.055751094"
Jul 06 01:26:37.156381 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:37.156350    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-chtsq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 01:26:37.156813 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:37.156406    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 01:26:40.156998 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.156963    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:40.753027 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.753001    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"]
Jul 06 01:26:40.756299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.756280    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.758622 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.758596    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 01:26:40.759236 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.759212    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-d79xn\""
Jul 06 01:26:40.759940 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.759918    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"]
Jul 06 01:26:40.763098 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.763075    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.765326 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.765310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wzfwm\""
Jul 06 01:26:40.765444 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.765423    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 01:26:40.770570 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.770549    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"]
Jul 06 01:26:40.781403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.781382    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"]
Jul 06 01:26:40.849305 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.849274    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25006f46-ee92-4f37-b393-5bad384d3483-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.849425 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.849364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m7bs\" (UniqueName: \"kubernetes.io/projected/c0eddcbf-0df4-43ce-927e-5e694d38f185-kube-api-access-7m7bs\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.849425 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.849395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpk6f\" (UniqueName: \"kubernetes.io/projected/25006f46-ee92-4f37-b393-5bad384d3483-kube-api-access-vpk6f\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.849425 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.849414    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0eddcbf-0df4-43ce-927e-5e694d38f185-cert\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.949942 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.949913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7bs\" (UniqueName: \"kubernetes.io/projected/c0eddcbf-0df4-43ce-927e-5e694d38f185-kube-api-access-7m7bs\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.950101 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.949950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpk6f\" (UniqueName: \"kubernetes.io/projected/25006f46-ee92-4f37-b393-5bad384d3483-kube-api-access-vpk6f\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.950101 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.949967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0eddcbf-0df4-43ce-927e-5e694d38f185-cert\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.950101 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.950020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25006f46-ee92-4f37-b393-5bad384d3483-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.952586 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.952562    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25006f46-ee92-4f37-b393-5bad384d3483-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.952695 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.952655    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0eddcbf-0df4-43ce-927e-5e694d38f185-cert\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:40.959147 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.959108    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpk6f\" (UniqueName: \"kubernetes.io/projected/25006f46-ee92-4f37-b393-5bad384d3483-kube-api-access-vpk6f\") pod \"llmisvc-controller-manager-777c6bd7d5-fj8fk\" (UID: \"25006f46-ee92-4f37-b393-5bad384d3483\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:40.959286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:40.959263    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7bs\" (UniqueName: \"kubernetes.io/projected/c0eddcbf-0df4-43ce-927e-5e694d38f185-kube-api-access-7m7bs\") pod \"kserve-controller-manager-d6f4b65b8-8xl24\" (UID: \"c0eddcbf-0df4-43ce-927e-5e694d38f185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:41.067229 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:41.067138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:41.075399 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:41.075378    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:41.237840 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:41.237760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"]
Jul 06 01:26:41.240615 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:41.240589    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0eddcbf_0df4_43ce_927e_5e694d38f185.slice/crio-2381f284f8b699f2d8365341016ec0a84de2a447d62812253bb2129793f4138f WatchSource:0}: Error finding container 2381f284f8b699f2d8365341016ec0a84de2a447d62812253bb2129793f4138f: Status 404 returned error can't find the container with id 2381f284f8b699f2d8365341016ec0a84de2a447d62812253bb2129793f4138f
Jul 06 01:26:41.258594 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:41.258545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"]
Jul 06 01:26:41.260406 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:26:41.260386    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25006f46_ee92_4f37_b393_5bad384d3483.slice/crio-3a9af8e6b74cf4853a176364f6b7e309267180ce56c4bfd40995b1edbf742600 WatchSource:0}: Error finding container 3a9af8e6b74cf4853a176364f6b7e309267180ce56c4bfd40995b1edbf742600: Status 404 returned error can't find the container with id 3a9af8e6b74cf4853a176364f6b7e309267180ce56c4bfd40995b1edbf742600
Jul 06 01:26:42.181237 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:42.181200    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24" event={"ID":"c0eddcbf-0df4-43ce-927e-5e694d38f185","Type":"ContainerStarted","Data":"2381f284f8b699f2d8365341016ec0a84de2a447d62812253bb2129793f4138f"}
Jul 06 01:26:42.182242 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:42.182222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk" event={"ID":"25006f46-ee92-4f37-b393-5bad384d3483","Type":"ContainerStarted","Data":"3a9af8e6b74cf4853a176364f6b7e309267180ce56c4bfd40995b1edbf742600"}
Jul 06 01:26:44.112210 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.112174    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:44.112767 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.112402    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery" containerID="cri-o://ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39" gracePeriod=30
Jul 06 01:26:44.500299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.500274    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:44.581364 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581331    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581527 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581386    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581592 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581575    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69gtc\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581656 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581621    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581709 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581668    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581709 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581703    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.581806 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.581737    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs\") pod \"bb05abb2-c979-4740-aca5-7b5e835cb617\" (UID: \"bb05abb2-c979-4740-aca5-7b5e835cb617\") "
Jul 06 01:26:44.582079 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.582048    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:26:44.583949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.583918    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts" (OuterVolumeSpecName: "cacerts") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:26:44.584036 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.583968    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs" (OuterVolumeSpecName: "local-certs") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:26:44.584195 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.584175    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:26:44.584279 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.584220    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc" (OuterVolumeSpecName: "kube-api-access-69gtc") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "kube-api-access-69gtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:26:44.584486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.584467    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:26:44.584538 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.584480    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "bb05abb2-c979-4740-aca5-7b5e835cb617" (UID: "bb05abb2-c979-4740-aca5-7b5e835cb617"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:26:44.682415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682387    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-dns-cert\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682409    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-token\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682419    2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb05abb2-c979-4740-aca5-7b5e835cb617-local-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682426    2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-kubeconfig\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682434    2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb05abb2-c979-4740-aca5-7b5e835cb617-cacerts\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682442    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69gtc\" (UniqueName: \"kubernetes.io/projected/bb05abb2-c979-4740-aca5-7b5e835cb617-kube-api-access-69gtc\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:44.682590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:44.682451    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb05abb2-c979-4740-aca5-7b5e835cb617-istio-csr-ca-configmap\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:26:45.197458 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.197414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24" event={"ID":"c0eddcbf-0df4-43ce-927e-5e694d38f185","Type":"ContainerStarted","Data":"fc47e489dca513187300c59d7c1009d6092c0d7cd6af40bccb7825ea7d06e64e"}
Jul 06 01:26:45.197879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.197488    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:26:45.198981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.198959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk" event={"ID":"25006f46-ee92-4f37-b393-5bad384d3483","Type":"ContainerStarted","Data":"8b8a7dea160aebe2c32142c848d3f5098c12d81210627ebfe560271495b560e3"}
Jul 06 01:26:45.199098 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.198998    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:26:45.200170 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.200151    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerID="ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39" exitCode=0
Jul 06 01:26:45.200257 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.200175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" event={"ID":"bb05abb2-c979-4740-aca5-7b5e835cb617","Type":"ContainerDied","Data":"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"}
Jul 06 01:26:45.200257 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.200188    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"
Jul 06 01:26:45.200257 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.200226    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-chtsq" event={"ID":"bb05abb2-c979-4740-aca5-7b5e835cb617","Type":"ContainerDied","Data":"c9cf8124dffbeecf017667d0567e9b054c82e01ec0e7c316f8840f95509aba47"}
Jul 06 01:26:45.200408 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.200257    2567 scope.go:117] "RemoveContainer" containerID="ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"
Jul 06 01:26:45.209836 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.209819    2567 scope.go:117] "RemoveContainer" containerID="ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"
Jul 06 01:26:45.210109 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:26:45.210089    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39\": container with ID starting with ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39 not found: ID does not exist" containerID="ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"
Jul 06 01:26:45.210252 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.210232    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39"} err="failed to get container status \"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39\": rpc error: code = NotFound desc = could not find container \"ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39\": container with ID starting with ca47bff7ae2aa9c30a7aa64155b7f824dd06fb74656d421bf97cc9f58ec5ec39 not found: ID does not exist"
Jul 06 01:26:45.216226 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.216150    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24" podStartSLOduration=2.091038358 podStartE2EDuration="5.216115693s" totalImagesPullingTime="3.125077335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:45.214402235 +0000 UTC m=+405.059551649" watchObservedRunningTime="2026-07-06 01:26:45.216115693 +0000 UTC m=+405.061265106"
Jul 06 01:26:45.233158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.233094    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk" podStartSLOduration=2.123433291 podStartE2EDuration="5.23308209s" totalImagesPullingTime="3.109648799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:45.231922188 +0000 UTC m=+405.077071620" watchObservedRunningTime="2026-07-06 01:26:45.23308209 +0000 UTC m=+405.078231526"
Jul 06 01:26:45.247941 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.247920    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:45.251594 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:45.251574    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-chtsq"]
Jul 06 01:26:46.656019 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:46.655985    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" path="/var/lib/kubelet/pods/bb05abb2-c979-4740-aca5-7b5e835cb617/volumes"
Jul 06 01:26:47.158557 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:47.158526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mqd8"
Jul 06 01:26:47.160550 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:26:47.160530    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pd7kl"
Jul 06 01:27:16.212851 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:16.212818    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fj8fk"
Jul 06 01:27:16.215867 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:16.215843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8xl24"
Jul 06 01:27:29.804842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.804765    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:29.805254 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.805075    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery"
Jul 06 01:27:29.805254 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.805087    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery"
Jul 06 01:27:29.805254 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.805170    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb05abb2-c979-4740-aca5-7b5e835cb617" containerName="discovery"
Jul 06 01:27:29.807196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.807174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:29.809501 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.809480    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 01:27:29.809617 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.809524    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2kvjb\""
Jul 06 01:27:29.810644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.810624    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 01:27:29.818186 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.818165    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:29.919286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:29.919254    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5l4\" (UniqueName: \"kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4\") pod \"kuadrant-operator-catalog-59wvw\" (UID: \"03798c73-d345-4b38-b2a4-7f7f8e0b503e\") " pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:30.020275 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.020241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5l4\" (UniqueName: \"kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4\") pod \"kuadrant-operator-catalog-59wvw\" (UID: \"03798c73-d345-4b38-b2a4-7f7f8e0b503e\") " pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:30.028209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.028181    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5l4\" (UniqueName: \"kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4\") pod \"kuadrant-operator-catalog-59wvw\" (UID: \"03798c73-d345-4b38-b2a4-7f7f8e0b503e\") " pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:30.117212 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.117184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:30.163038 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.162983    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:30.246104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.246077    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:30.250384 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:27:30.250353    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03798c73_d345_4b38_b2a4_7f7f8e0b503e.slice/crio-ff6c486dfda5c1134bd79fd9adad7f043ebac687ce5f42c0e6f942ddb7fcfbba WatchSource:0}: Error finding container ff6c486dfda5c1134bd79fd9adad7f043ebac687ce5f42c0e6f942ddb7fcfbba: Status 404 returned error can't find the container with id ff6c486dfda5c1134bd79fd9adad7f043ebac687ce5f42c0e6f942ddb7fcfbba
Jul 06 01:27:30.372112 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.372035    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xnrhn"]
Jul 06 01:27:30.374619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.374604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:30.382393 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.382369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xnrhn"]
Jul 06 01:27:30.388181 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.388150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" event={"ID":"03798c73-d345-4b38-b2a4-7f7f8e0b503e","Type":"ContainerStarted","Data":"ff6c486dfda5c1134bd79fd9adad7f043ebac687ce5f42c0e6f942ddb7fcfbba"}
Jul 06 01:27:30.525822 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.525793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbpl\" (UniqueName: \"kubernetes.io/projected/141cee8e-3fc7-4ba4-ba1b-67040c43c3b8-kube-api-access-lpbpl\") pod \"kuadrant-operator-catalog-xnrhn\" (UID: \"141cee8e-3fc7-4ba4-ba1b-67040c43c3b8\") " pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:30.627083 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.627009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbpl\" (UniqueName: \"kubernetes.io/projected/141cee8e-3fc7-4ba4-ba1b-67040c43c3b8-kube-api-access-lpbpl\") pod \"kuadrant-operator-catalog-xnrhn\" (UID: \"141cee8e-3fc7-4ba4-ba1b-67040c43c3b8\") " pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:30.634825 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.634798    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbpl\" (UniqueName: \"kubernetes.io/projected/141cee8e-3fc7-4ba4-ba1b-67040c43c3b8-kube-api-access-lpbpl\") pod \"kuadrant-operator-catalog-xnrhn\" (UID: \"141cee8e-3fc7-4ba4-ba1b-67040c43c3b8\") " pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:30.684951 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.684926    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:30.815625 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:30.815600    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xnrhn"]
Jul 06 01:27:30.817159 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:27:30.817101    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141cee8e_3fc7_4ba4_ba1b_67040c43c3b8.slice/crio-9b804be237b49294dd9702878af2ffe9cee9f007f4a2f5f42a5516777d32229b WatchSource:0}: Error finding container 9b804be237b49294dd9702878af2ffe9cee9f007f4a2f5f42a5516777d32229b: Status 404 returned error can't find the container with id 9b804be237b49294dd9702878af2ffe9cee9f007f4a2f5f42a5516777d32229b
Jul 06 01:27:31.394383 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:31.394349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn" event={"ID":"141cee8e-3fc7-4ba4-ba1b-67040c43c3b8","Type":"ContainerStarted","Data":"9b804be237b49294dd9702878af2ffe9cee9f007f4a2f5f42a5516777d32229b"}
Jul 06 01:27:33.405471 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.405440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" event={"ID":"03798c73-d345-4b38-b2a4-7f7f8e0b503e","Type":"ContainerStarted","Data":"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"}
Jul 06 01:27:33.405929 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.405487    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" podUID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" containerName="registry-server" containerID="cri-o://06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c" gracePeriod=2
Jul 06 01:27:33.406902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.406877    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn" event={"ID":"141cee8e-3fc7-4ba4-ba1b-67040c43c3b8","Type":"ContainerStarted","Data":"8a9fc2e46cbb3a77d4a9317df16d662b4cdf00828df81e4928b03279a4519841"}
Jul 06 01:27:33.419840 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.419799    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" podStartSLOduration=2.148271304 podStartE2EDuration="4.419784777s" totalImagesPullingTime="2.271513473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:33.419034601 +0000 UTC m=+453.264184012" watchObservedRunningTime="2026-07-06 01:27:33.419784777 +0000 UTC m=+453.264934189"
Jul 06 01:27:33.432757 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.432717    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn" podStartSLOduration=1.7304555640000001 podStartE2EDuration="3.432702254s" totalImagesPullingTime="1.70224669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:33.431873972 +0000 UTC m=+453.277023385" watchObservedRunningTime="2026-07-06 01:27:33.432702254 +0000 UTC m=+453.277851665"
Jul 06 01:27:33.645018 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.644997    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:33.752115 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.752044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t5l4\" (UniqueName: \"kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4\") pod \"03798c73-d345-4b38-b2a4-7f7f8e0b503e\" (UID: \"03798c73-d345-4b38-b2a4-7f7f8e0b503e\") "
Jul 06 01:27:33.754271 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.754250    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4" (OuterVolumeSpecName: "kube-api-access-6t5l4") pod "03798c73-d345-4b38-b2a4-7f7f8e0b503e" (UID: "03798c73-d345-4b38-b2a4-7f7f8e0b503e"). InnerVolumeSpecName "kube-api-access-6t5l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:27:33.853456 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:33.853417    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t5l4\" (UniqueName: \"kubernetes.io/projected/03798c73-d345-4b38-b2a4-7f7f8e0b503e-kube-api-access-6t5l4\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:27:34.411948 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.411912    2567 generic.go:356] "Generic (PLEG): container finished" podID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" containerID="06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c" exitCode=0
Jul 06 01:27:34.412409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.411955    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-59wvw"
Jul 06 01:27:34.412409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.411996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" event={"ID":"03798c73-d345-4b38-b2a4-7f7f8e0b503e","Type":"ContainerDied","Data":"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"}
Jul 06 01:27:34.412409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.412040    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-59wvw" event={"ID":"03798c73-d345-4b38-b2a4-7f7f8e0b503e","Type":"ContainerDied","Data":"ff6c486dfda5c1134bd79fd9adad7f043ebac687ce5f42c0e6f942ddb7fcfbba"}
Jul 06 01:27:34.412409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.412060    2567 scope.go:117] "RemoveContainer" containerID="06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"
Jul 06 01:27:34.421048 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.421035    2567 scope.go:117] "RemoveContainer" containerID="06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"
Jul 06 01:27:34.421318 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:27:34.421300    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c\": container with ID starting with 06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c not found: ID does not exist" containerID="06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"
Jul 06 01:27:34.421360 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.421326    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c"} err="failed to get container status \"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c\": rpc error: code = NotFound desc = could not find container \"06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c\": container with ID starting with 06f3bbb1edbfc14b986f79810aa109aeae6a8978896e117944a6626931a4032c not found: ID does not exist"
Jul 06 01:27:34.431714 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.431690    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:34.433423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.433404    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-59wvw"]
Jul 06 01:27:34.655904 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:34.655873    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" path="/var/lib/kubelet/pods/03798c73-d345-4b38-b2a4-7f7f8e0b503e/volumes"
Jul 06 01:27:40.685049 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:40.685021    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:40.685454 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:40.685075    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:40.707330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:40.707305    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:27:41.463418 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:27:41.463387    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xnrhn"
Jul 06 01:28:01.690309 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.690277    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"]
Jul 06 01:28:01.690789 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.690692    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" containerName="registry-server"
Jul 06 01:28:01.690789 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.690709    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" containerName="registry-server"
Jul 06 01:28:01.690909 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.690811    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="03798c73-d345-4b38-b2a4-7f7f8e0b503e" containerName="registry-server"
Jul 06 01:28:01.694848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.694829    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:01.698690 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.698656    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 01:28:01.698800 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.698712    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nvr98\""
Jul 06 01:28:01.712399 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.712376    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"]
Jul 06 01:28:01.768384 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.768352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzkg7\" (UniqueName: \"kubernetes.io/projected/62188051-0e95-44e6-ad1d-138fa92f4399-kube-api-access-nzkg7\") pod \"dns-operator-controller-manager-864dd78c4c-9zmvj\" (UID: \"62188051-0e95-44e6-ad1d-138fa92f4399\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:01.869380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.869349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzkg7\" (UniqueName: \"kubernetes.io/projected/62188051-0e95-44e6-ad1d-138fa92f4399-kube-api-access-nzkg7\") pod \"dns-operator-controller-manager-864dd78c4c-9zmvj\" (UID: \"62188051-0e95-44e6-ad1d-138fa92f4399\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:01.881960 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:01.881941    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzkg7\" (UniqueName: \"kubernetes.io/projected/62188051-0e95-44e6-ad1d-138fa92f4399-kube-api-access-nzkg7\") pod \"dns-operator-controller-manager-864dd78c4c-9zmvj\" (UID: \"62188051-0e95-44e6-ad1d-138fa92f4399\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:02.005730 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:02.005659    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:02.136076 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:02.136047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"]
Jul 06 01:28:02.139845 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:02.139818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62188051_0e95_44e6_ad1d_138fa92f4399.slice/crio-3ee3fc6e6489c0e02ca33cb01e0b9cbb466b3ed54e4207222484e1b1e64a24ec WatchSource:0}: Error finding container 3ee3fc6e6489c0e02ca33cb01e0b9cbb466b3ed54e4207222484e1b1e64a24ec: Status 404 returned error can't find the container with id 3ee3fc6e6489c0e02ca33cb01e0b9cbb466b3ed54e4207222484e1b1e64a24ec
Jul 06 01:28:02.522665 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:02.522627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj" event={"ID":"62188051-0e95-44e6-ad1d-138fa92f4399","Type":"ContainerStarted","Data":"3ee3fc6e6489c0e02ca33cb01e0b9cbb466b3ed54e4207222484e1b1e64a24ec"}
Jul 06 01:28:04.533491 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:04.533458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj" event={"ID":"62188051-0e95-44e6-ad1d-138fa92f4399","Type":"ContainerStarted","Data":"49b7ea3cea48e84a574aedf1a74b312cb37fe88fb4610a73d5a8af83461e0d83"}
Jul 06 01:28:04.533863 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:04.533585    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:04.548586 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:04.548544    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj" podStartSLOduration=1.616461242 podStartE2EDuration="3.548531589s" totalImagesPullingTime="1.932070347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:04.547761787 +0000 UTC m=+484.392911234" watchObservedRunningTime="2026-07-06 01:28:04.548531589 +0000 UTC m=+484.393681064"
Jul 06 01:28:05.123947 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.123913    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"]
Jul 06 01:28:05.127268 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.127250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:05.129532 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.129514    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jckrs\""
Jul 06 01:28:05.136783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.136763    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"]
Jul 06 01:28:05.196145 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.196088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshzs\" (UniqueName: \"kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs\") pod \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" (UID: \"3ed381b0-2438-43ea-97f6-64402a419350\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:05.297056 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.297020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshzs\" (UniqueName: \"kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs\") pod \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" (UID: \"3ed381b0-2438-43ea-97f6-64402a419350\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:05.311100 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.311071    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshzs\" (UniqueName: \"kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs\") pod \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" (UID: \"3ed381b0-2438-43ea-97f6-64402a419350\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:05.437677 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.437602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:05.568802 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:05.568774    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"]
Jul 06 01:28:05.572241 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:05.572216    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed381b0_2438_43ea_97f6_64402a419350.slice/crio-6e27dea0a7aa9e9b740b9cb1dfdd6d4b109307622eeb1a986be02183d1796a83 WatchSource:0}: Error finding container 6e27dea0a7aa9e9b740b9cb1dfdd6d4b109307622eeb1a986be02183d1796a83: Status 404 returned error can't find the container with id 6e27dea0a7aa9e9b740b9cb1dfdd6d4b109307622eeb1a986be02183d1796a83
Jul 06 01:28:06.545544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:06.545503    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd" event={"ID":"3ed381b0-2438-43ea-97f6-64402a419350","Type":"ContainerStarted","Data":"6e27dea0a7aa9e9b740b9cb1dfdd6d4b109307622eeb1a986be02183d1796a83"}
Jul 06 01:28:07.550890 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:07.550814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd" event={"ID":"3ed381b0-2438-43ea-97f6-64402a419350","Type":"ContainerStarted","Data":"c0220fa592dd9329d353c8e53d741271a75b3fd1b934d2ce8154daa5105ca70e"}
Jul 06 01:28:07.550890 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:07.550878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:07.570224 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:07.570181    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd" podStartSLOduration=0.943156612 podStartE2EDuration="2.570165306s" totalImagesPullingTime="1.627008694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:07.568490694 +0000 UTC m=+487.413640124" watchObservedRunningTime="2026-07-06 01:28:07.570165306 +0000 UTC m=+487.415314717"
Jul 06 01:28:10.139319 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.139283    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"]
Jul 06 01:28:10.142756 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.142742    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.145255 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.145234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v2k2b\""
Jul 06 01:28:10.156936 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.156912    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"]
Jul 06 01:28:10.232653 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.232618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.232653 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.232660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rcwp\" (UniqueName: \"kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.333678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.333644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.333678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.333683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rcwp\" (UniqueName: \"kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.334068 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.334048    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.342101 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.342071    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rcwp\" (UniqueName: \"kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.452459 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.452380    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:10.589811 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:10.589788    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"]
Jul 06 01:28:10.592619 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:10.592590    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bd0dbf_f547_4658_ac05_48098bc03794.slice/crio-736c23296ae2efe3de6b4d753dd6986c00b53bc108b416e60f56f5f773ccca51 WatchSource:0}: Error finding container 736c23296ae2efe3de6b4d753dd6986c00b53bc108b416e60f56f5f773ccca51: Status 404 returned error can't find the container with id 736c23296ae2efe3de6b4d753dd6986c00b53bc108b416e60f56f5f773ccca51
Jul 06 01:28:11.568416 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:11.568378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb" event={"ID":"82bd0dbf-f547-4658-ac05-48098bc03794","Type":"ContainerStarted","Data":"736c23296ae2efe3de6b4d753dd6986c00b53bc108b416e60f56f5f773ccca51"}
Jul 06 01:28:13.231855 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.231824    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"]
Jul 06 01:28:13.236710 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.236685    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:13.239022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.238996    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rq9xd\""
Jul 06 01:28:13.242651 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.242619    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"]
Jul 06 01:28:13.258941 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.258906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97rrw\" (UniqueName: \"kubernetes.io/projected/37040ac5-5b89-458e-a170-d8c2ff75e594-kube-api-access-97rrw\") pod \"authorino-operator-75cc8d5cb7-fsf4h\" (UID: \"37040ac5-5b89-458e-a170-d8c2ff75e594\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:13.359697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.359647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97rrw\" (UniqueName: \"kubernetes.io/projected/37040ac5-5b89-458e-a170-d8c2ff75e594-kube-api-access-97rrw\") pod \"authorino-operator-75cc8d5cb7-fsf4h\" (UID: \"37040ac5-5b89-458e-a170-d8c2ff75e594\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:13.370511 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.370481    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97rrw\" (UniqueName: \"kubernetes.io/projected/37040ac5-5b89-458e-a170-d8c2ff75e594-kube-api-access-97rrw\") pod \"authorino-operator-75cc8d5cb7-fsf4h\" (UID: \"37040ac5-5b89-458e-a170-d8c2ff75e594\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:13.550605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.550522    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:13.705379 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:13.705136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"]
Jul 06 01:28:13.707650 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:13.707625    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37040ac5_5b89_458e_a170_d8c2ff75e594.slice/crio-a7692ca36990440685c57ed2fc9e564ec80b5d181d2d93cd88e23da79c4e6ce7 WatchSource:0}: Error finding container a7692ca36990440685c57ed2fc9e564ec80b5d181d2d93cd88e23da79c4e6ce7: Status 404 returned error can't find the container with id a7692ca36990440685c57ed2fc9e564ec80b5d181d2d93cd88e23da79c4e6ce7
Jul 06 01:28:14.586061 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:14.585995    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h" event={"ID":"37040ac5-5b89-458e-a170-d8c2ff75e594","Type":"ContainerStarted","Data":"a7692ca36990440685c57ed2fc9e564ec80b5d181d2d93cd88e23da79c4e6ce7"}
Jul 06 01:28:15.541807 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:15.541778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zmvj"
Jul 06 01:28:16.597389 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.597358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h" event={"ID":"37040ac5-5b89-458e-a170-d8c2ff75e594","Type":"ContainerStarted","Data":"2aa6038b3974502475bd4ffe10f8100f85fca77f7f90a6cf8e85b2d3467e8a97"}
Jul 06 01:28:16.597785 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.597464    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:16.599012 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.598990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb" event={"ID":"82bd0dbf-f547-4658-ac05-48098bc03794","Type":"ContainerStarted","Data":"e4150ab280fe4b6c763e669b9ad5fff1180aead309a78a12a6f3b4eee4221b1a"}
Jul 06 01:28:16.599106 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.599029    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:16.618763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.618717    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h" podStartSLOduration=0.824938568 podStartE2EDuration="3.618701781s" totalImagesPullingTime="2.793763213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:16.616737457 +0000 UTC m=+496.461886893" watchObservedRunningTime="2026-07-06 01:28:16.618701781 +0000 UTC m=+496.463851193"
Jul 06 01:28:16.634350 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:16.634308    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb" podStartSLOduration=0.763900625 podStartE2EDuration="6.634292213s" totalImagesPullingTime="5.870391588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:16.633142721 +0000 UTC m=+496.478292144" watchObservedRunningTime="2026-07-06 01:28:16.634292213 +0000 UTC m=+496.479441625"
Jul 06 01:28:18.557087 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:18.557056    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:19.347925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.347892    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"]
Jul 06 01:28:19.351631 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.351609    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.353846 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.353823    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 06 01:28:19.354027 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.354008    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 06 01:28:19.354137 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.354105    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wsd9j\""
Jul 06 01:28:19.359404 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.359352    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"]
Jul 06 01:28:19.409537 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.409503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.409738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.409562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnzk5\" (UniqueName: \"kubernetes.io/projected/d1411357-0205-495f-8ac2-28472483e807-kube-api-access-bnzk5\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.409738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.409617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1411357-0205-495f-8ac2-28472483e807-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.510469 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.510435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.510630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.510499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnzk5\" (UniqueName: \"kubernetes.io/projected/d1411357-0205-495f-8ac2-28472483e807-kube-api-access-bnzk5\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.510630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.510527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1411357-0205-495f-8ac2-28472483e807-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.510630 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:19.510589    2567 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 06 01:28:19.510820 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:19.510672    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert podName:d1411357-0205-495f-8ac2-28472483e807 nodeName:}" failed. No retries permitted until 2026-07-06 01:28:20.010651129 +0000 UTC m=+499.855800522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xlghp" (UID: "d1411357-0205-495f-8ac2-28472483e807") : secret "plugin-serving-cert" not found
Jul 06 01:28:19.511196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.511178    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1411357-0205-495f-8ac2-28472483e807-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:19.518031 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:19.518003    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnzk5\" (UniqueName: \"kubernetes.io/projected/d1411357-0205-495f-8ac2-28472483e807-kube-api-access-bnzk5\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:20.013438 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:20.013402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:20.015925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:20.015898    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1411357-0205-495f-8ac2-28472483e807-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xlghp\" (UID: \"d1411357-0205-495f-8ac2-28472483e807\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:20.261332 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:20.261297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"
Jul 06 01:28:20.400029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:20.399954    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp"]
Jul 06 01:28:20.402224 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:20.402196    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1411357_0205_495f_8ac2_28472483e807.slice/crio-6caed3080b7fa097175079c5cab276d55c462b232ecd558e138089ec87528127 WatchSource:0}: Error finding container 6caed3080b7fa097175079c5cab276d55c462b232ecd558e138089ec87528127: Status 404 returned error can't find the container with id 6caed3080b7fa097175079c5cab276d55c462b232ecd558e138089ec87528127
Jul 06 01:28:20.616977 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:20.616950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp" event={"ID":"d1411357-0205-495f-8ac2-28472483e807","Type":"ContainerStarted","Data":"6caed3080b7fa097175079c5cab276d55c462b232ecd558e138089ec87528127"}
Jul 06 01:28:27.605813 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:27.605778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fsf4h"
Jul 06 01:28:27.606305 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:27.605861    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:28.848516 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:28.848471    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"]
Jul 06 01:28:28.858352 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:28.858325    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:28.864859 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:28.864822    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"]
Jul 06 01:28:28.997238 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:28.997204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:28.997408 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:28.997252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zjx4\" (UniqueName: \"kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.098423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.098395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.098624 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.098438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjx4\" (UniqueName: \"kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.098845 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.098823    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.113955 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.113926    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjx4\" (UniqueName: \"kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4\") pod \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.172392 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.172356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:29.255989 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.255957    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"]
Jul 06 01:28:29.260031 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.260009    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"]
Jul 06 01:28:29.278188 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.278154    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:28:29.285403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.285379    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"]
Jul 06 01:28:29.285733 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.285569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.285733 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.285565    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" containerName="manager" containerID="cri-o://e4150ab280fe4b6c763e669b9ad5fff1180aead309a78a12a6f3b4eee4221b1a" gracePeriod=2
Jul 06 01:28:29.296949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.296049    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:28:29.300540 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.300477    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"]
Jul 06 01:28:29.307863 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.307435    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"82bd0dbf-f547-4658-ac05-48098bc03794\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\": pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:29.309507 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.309471    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:29.313666 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.313638    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"]
Jul 06 01:28:29.314092 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.314047    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd" podUID="3ed381b0-2438-43ea-97f6-64402a419350" containerName="manager" containerID="cri-o://c0220fa592dd9329d353c8e53d741271a75b3fd1b934d2ce8154daa5105ca70e" gracePeriod=2
Jul 06 01:28:29.317133 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.317091    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"]
Jul 06 01:28:29.331627 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.331590    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:29.336213 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.336157    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:29.337209 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337185    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"]
Jul 06 01:28:29.337592 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337577    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" containerName="manager"
Jul 06 01:28:29.337689 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337595    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" containerName="manager"
Jul 06 01:28:29.337689 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337614    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed381b0-2438-43ea-97f6-64402a419350" containerName="manager"
Jul 06 01:28:29.337689 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337622    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed381b0-2438-43ea-97f6-64402a419350" containerName="manager"
Jul 06 01:28:29.337840 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337703    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed381b0-2438-43ea-97f6-64402a419350" containerName="manager"
Jul 06 01:28:29.337840 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.337715    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" containerName="manager"
Jul 06 01:28:29.342258 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.342238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:29.344461 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.344425    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:29.346355 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:29.346325    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:29.351215 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.351147    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"]
Jul 06 01:28:29.401400 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.401373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ztxd\" (UniqueName: \"kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.401575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.401426    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.401650 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.401608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xk64\" (UniqueName: \"kubernetes.io/projected/671b3910-20c4-4407-b87b-59ee49dcf1e9-kube-api-access-4xk64\") pod \"limitador-operator-controller-manager-859d7b78bd-gnnxr\" (UID: \"671b3910-20c4-4407-b87b-59ee49dcf1e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:29.503066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.503030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xk64\" (UniqueName: \"kubernetes.io/projected/671b3910-20c4-4407-b87b-59ee49dcf1e9-kube-api-access-4xk64\") pod \"limitador-operator-controller-manager-859d7b78bd-gnnxr\" (UID: \"671b3910-20c4-4407-b87b-59ee49dcf1e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:29.503066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.503075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ztxd\" (UniqueName: \"kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.503340 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.503113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.503529 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.503502    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.524450 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.524421    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ztxd\" (UniqueName: \"kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd\") pod \"kuadrant-operator-controller-manager-55467d9f7-ghqp5\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.525936 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.525909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xk64\" (UniqueName: \"kubernetes.io/projected/671b3910-20c4-4407-b87b-59ee49dcf1e9-kube-api-access-4xk64\") pod \"limitador-operator-controller-manager-859d7b78bd-gnnxr\" (UID: \"671b3910-20c4-4407-b87b-59ee49dcf1e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:29.604187 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.604084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:29.727496 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:29.727466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:30.654774 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:30.654735    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:30.656498 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:30.656470    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:41.736807 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.736757    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ed381b0-2438-43ea-97f6-64402a419350" containerID="c0220fa592dd9329d353c8e53d741271a75b3fd1b934d2ce8154daa5105ca70e" exitCode=0
Jul 06 01:28:41.738535 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.738505    2567 generic.go:356] "Generic (PLEG): container finished" podID="82bd0dbf-f547-4658-ac05-48098bc03794" containerID="e4150ab280fe4b6c763e669b9ad5fff1180aead309a78a12a6f3b4eee4221b1a" exitCode=0
Jul 06 01:28:41.767410 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.767387    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:41.769302 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:41.769274    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:41.775567 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.775502    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"]
Jul 06 01:28:41.781567 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:41.781538    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671b3910_20c4_4407_b87b_59ee49dcf1e9.slice/crio-d4e95f74b8b3f52e576a76c992784ddd61709f04a2a34791e495430f762ab15d WatchSource:0}: Error finding container d4e95f74b8b3f52e576a76c992784ddd61709f04a2a34791e495430f762ab15d: Status 404 returned error can't find the container with id d4e95f74b8b3f52e576a76c992784ddd61709f04a2a34791e495430f762ab15d
Jul 06 01:28:41.784369 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.784331    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:41.786075 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:41.786050    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:41.787880 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:41.787767    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:41.803495 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.803477    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:28:41.810209 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:41.810189    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0064db_dd09_4fe8_ad21_f9b788097992.slice/crio-3e7a53412c7b1f5aabe6f3bbdc6673b78fa5f3bd6f26edf21abf2597ae3dae67 WatchSource:0}: Error finding container 3e7a53412c7b1f5aabe6f3bbdc6673b78fa5f3bd6f26edf21abf2597ae3dae67: Status 404 returned error can't find the container with id 3e7a53412c7b1f5aabe6f3bbdc6673b78fa5f3bd6f26edf21abf2597ae3dae67
Jul 06 01:28:41.920606 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.920574    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tshzs\" (UniqueName: \"kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs\") pod \"3ed381b0-2438-43ea-97f6-64402a419350\" (UID: \"3ed381b0-2438-43ea-97f6-64402a419350\") "
Jul 06 01:28:41.920794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.920622    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rcwp\" (UniqueName: \"kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp\") pod \"82bd0dbf-f547-4658-ac05-48098bc03794\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") "
Jul 06 01:28:41.920794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.920689    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume\") pod \"82bd0dbf-f547-4658-ac05-48098bc03794\" (UID: \"82bd0dbf-f547-4658-ac05-48098bc03794\") "
Jul 06 01:28:41.921502 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.921471    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "82bd0dbf-f547-4658-ac05-48098bc03794" (UID: "82bd0dbf-f547-4658-ac05-48098bc03794"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:28:41.922796 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.922771    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs" (OuterVolumeSpecName: "kube-api-access-tshzs") pod "3ed381b0-2438-43ea-97f6-64402a419350" (UID: "3ed381b0-2438-43ea-97f6-64402a419350"). InnerVolumeSpecName "kube-api-access-tshzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:28:41.922968 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:41.922944    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp" (OuterVolumeSpecName: "kube-api-access-4rcwp") pod "82bd0dbf-f547-4658-ac05-48098bc03794" (UID: "82bd0dbf-f547-4658-ac05-48098bc03794"). InnerVolumeSpecName "kube-api-access-4rcwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:28:42.021805 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.021775    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tshzs\" (UniqueName: \"kubernetes.io/projected/3ed381b0-2438-43ea-97f6-64402a419350-kube-api-access-tshzs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:28:42.021805 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.021799    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rcwp\" (UniqueName: \"kubernetes.io/projected/82bd0dbf-f547-4658-ac05-48098bc03794-kube-api-access-4rcwp\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:28:42.021805 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.021809    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bd0dbf-f547-4658-ac05-48098bc03794-extensions-socket-volume\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:28:42.656452 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.656422    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed381b0-2438-43ea-97f6-64402a419350" path="/var/lib/kubelet/pods/3ed381b0-2438-43ea-97f6-64402a419350/volumes"
Jul 06 01:28:42.656763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.656750    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" path="/var/lib/kubelet/pods/82bd0dbf-f547-4658-ac05-48098bc03794/volumes"
Jul 06 01:28:42.743980 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.743955    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:42.744418 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.743988    2567 scope.go:117] "RemoveContainer" containerID="c0220fa592dd9329d353c8e53d741271a75b3fd1b934d2ce8154daa5105ca70e"
Jul 06 01:28:42.745474 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.745453    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:42.746105 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.746023    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:42.747375 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.747351    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp" event={"ID":"d1411357-0205-495f-8ac2-28472483e807","Type":"ContainerStarted","Data":"8334c175026613bbcbd8ebcbe95d92006f93305d5d4a3e38cba3e14cad38781f"}
Jul 06 01:28:42.747815 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.747789    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:42.749078 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.749054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr" event={"ID":"671b3910-20c4-4407-b87b-59ee49dcf1e9","Type":"ContainerStarted","Data":"10bdb989cb188896bca0e3c193869330e894dec94b579ec7d9052f184bc8954e"}
Jul 06 01:28:42.749234 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.749095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr" event={"ID":"671b3910-20c4-4407-b87b-59ee49dcf1e9","Type":"ContainerStarted","Data":"d4e95f74b8b3f52e576a76c992784ddd61709f04a2a34791e495430f762ab15d"}
Jul 06 01:28:42.749234 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.749144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:42.749919 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.749884    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:42.751786 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.751752    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:42.753556 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.753528    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:42.756254 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.756234    2567 scope.go:117] "RemoveContainer" containerID="e4150ab280fe4b6c763e669b9ad5fff1180aead309a78a12a6f3b4eee4221b1a"
Jul 06 01:28:42.756575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.756550    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" event={"ID":"4d0064db-dd09-4fe8-ad21-f9b788097992","Type":"ContainerStarted","Data":"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"}
Jul 06 01:28:42.756656 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.756627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" event={"ID":"4d0064db-dd09-4fe8-ad21-f9b788097992","Type":"ContainerStarted","Data":"3e7a53412c7b1f5aabe6f3bbdc6673b78fa5f3bd6f26edf21abf2597ae3dae67"}
Jul 06 01:28:42.756709 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.756656    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:42.771302 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.771256    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr" podStartSLOduration=13.771243166 podStartE2EDuration="13.771243166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:42.770326578 +0000 UTC m=+522.615476015" watchObservedRunningTime="2026-07-06 01:28:42.771243166 +0000 UTC m=+522.616392577"
Jul 06 01:28:42.785759 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.785723    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xlghp" podStartSLOduration=2.50023519 podStartE2EDuration="23.785711144s" totalImagesPullingTime="21.285475954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:42.784500104 +0000 UTC m=+522.629649542" watchObservedRunningTime="2026-07-06 01:28:42.785711144 +0000 UTC m=+522.630860557"
Jul 06 01:28:42.786366 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.786343    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bf9cb\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="82bd0dbf-f547-4658-ac05-48098bc03794" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bf9cb"
Jul 06 01:28:42.787921 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:42.787899    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hnzhd\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="3ed381b0-2438-43ea-97f6-64402a419350" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnzhd"
Jul 06 01:28:42.810379 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:42.810339    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" podStartSLOduration=13.810325081 podStartE2EDuration="13.810325081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:42.808801191 +0000 UTC m=+522.653950623" watchObservedRunningTime="2026-07-06 01:28:42.810325081 +0000 UTC m=+522.655474492"
Jul 06 01:28:44.139222 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:44.139185    2567 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-b6z5r_kuadrant-system_8eb95d78-75d2-49e8-ad3c-45283bddaaef_0(54f9e2410712983938c31436aa06edd08c0394757c5d89408c657e8c6bceeb6f): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-b6z5r to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 06 01:28:44.139486 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:44.139251    2567 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-b6z5r_kuadrant-system_8eb95d78-75d2-49e8-ad3c-45283bddaaef_0(54f9e2410712983938c31436aa06edd08c0394757c5d89408c657e8c6bceeb6f): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-b6z5r to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:44.767001 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.766973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:44.768957 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:44.768924    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="8eb95d78-75d2-49e8-ad3c-45283bddaaef" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:44.776479 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.776460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:44.778441 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:44.778420    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="8eb95d78-75d2-49e8-ad3c-45283bddaaef" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:44.946436 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.946410    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zjx4\" (UniqueName: \"kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4\") pod \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") "
Jul 06 01:28:44.946594 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.946469    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume\") pod \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\" (UID: \"8eb95d78-75d2-49e8-ad3c-45283bddaaef\") "
Jul 06 01:28:44.946763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.946744    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8eb95d78-75d2-49e8-ad3c-45283bddaaef" (UID: "8eb95d78-75d2-49e8-ad3c-45283bddaaef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:28:44.948647 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:44.948628    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4" (OuterVolumeSpecName: "kube-api-access-5zjx4") pod "8eb95d78-75d2-49e8-ad3c-45283bddaaef" (UID: "8eb95d78-75d2-49e8-ad3c-45283bddaaef"). InnerVolumeSpecName "kube-api-access-5zjx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:28:45.047668 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:45.047605    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zjx4\" (UniqueName: \"kubernetes.io/projected/8eb95d78-75d2-49e8-ad3c-45283bddaaef-kube-api-access-5zjx4\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:28:45.047668 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:45.047629    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8eb95d78-75d2-49e8-ad3c-45283bddaaef-extensions-socket-volume\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:28:45.771190 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:45.771160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:45.773292 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:45.773256    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="8eb95d78-75d2-49e8-ad3c-45283bddaaef" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:45.778959 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:28:45.778929    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b6z5r\" is forbidden: User \"system:node:ip-10-0-133-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-137.ec2.internal' and this object" podUID="8eb95d78-75d2-49e8-ad3c-45283bddaaef" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b6z5r"
Jul 06 01:28:46.656394 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:46.656364    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eb95d78-75d2-49e8-ad3c-45283bddaaef" path="/var/lib/kubelet/pods/8eb95d78-75d2-49e8-ad3c-45283bddaaef/volumes"
Jul 06 01:28:53.765056 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:53.764981    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:28:53.765515 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:53.765157    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gnnxr"
Jul 06 01:28:54.274212 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.274181    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:28:54.277294 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.277279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.290144 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.290105    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:28:54.314753 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.314727    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.314867 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.314756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8slw8\" (UniqueName: \"kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.415873 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.415846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.415985 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.415877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8slw8\" (UniqueName: \"kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.416240 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.416223    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.423702 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.423678    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8slw8\" (UniqueName: \"kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8\") pod \"kuadrant-operator-controller-manager-86878c769f-kh7rl\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.587324 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.587293    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.723218 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.723189    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:28:54.725826 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:28:54.725793    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c0f473_a7b3_4658_8ff2_1b196941d6a2.slice/crio-c89861493ac16a6f8db488e964215dbafccc6c73258ce6e9d8cbd9495910972e WatchSource:0}: Error finding container c89861493ac16a6f8db488e964215dbafccc6c73258ce6e9d8cbd9495910972e: Status 404 returned error can't find the container with id c89861493ac16a6f8db488e964215dbafccc6c73258ce6e9d8cbd9495910972e
Jul 06 01:28:54.813998 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.813973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" event={"ID":"46c0f473-a7b3-4658-8ff2-1b196941d6a2","Type":"ContainerStarted","Data":"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"}
Jul 06 01:28:54.814329 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.814020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" event={"ID":"46c0f473-a7b3-4658-8ff2-1b196941d6a2","Type":"ContainerStarted","Data":"c89861493ac16a6f8db488e964215dbafccc6c73258ce6e9d8cbd9495910972e"}
Jul 06 01:28:54.814329 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.814050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:28:54.834494 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:28:54.834449    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" podStartSLOduration=0.83443541 podStartE2EDuration="834.43541ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:54.833236375 +0000 UTC m=+534.678385812" watchObservedRunningTime="2026-07-06 01:28:54.83443541 +0000 UTC m=+534.679584821"
Jul 06 01:29:05.821572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:05.821542    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:29:05.869389 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:05.869357    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:29:05.869596 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:05.869555    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" podUID="4d0064db-dd09-4fe8-ad21-f9b788097992" containerName="manager" containerID="cri-o://6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219" gracePeriod=10
Jul 06 01:29:06.117943 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.117923    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:29:06.194114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.194084    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume\") pod \"4d0064db-dd09-4fe8-ad21-f9b788097992\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") "
Jul 06 01:29:06.194290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.194147    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ztxd\" (UniqueName: \"kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd\") pod \"4d0064db-dd09-4fe8-ad21-f9b788097992\" (UID: \"4d0064db-dd09-4fe8-ad21-f9b788097992\") "
Jul 06 01:29:06.194435 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.194411    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4d0064db-dd09-4fe8-ad21-f9b788097992" (UID: "4d0064db-dd09-4fe8-ad21-f9b788097992"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:29:06.196359 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.196337    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd" (OuterVolumeSpecName: "kube-api-access-8ztxd") pod "4d0064db-dd09-4fe8-ad21-f9b788097992" (UID: "4d0064db-dd09-4fe8-ad21-f9b788097992"). InnerVolumeSpecName "kube-api-access-8ztxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:29:06.295018 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.294988    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d0064db-dd09-4fe8-ad21-f9b788097992-extensions-socket-volume\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:29:06.295018 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.295012    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ztxd\" (UniqueName: \"kubernetes.io/projected/4d0064db-dd09-4fe8-ad21-f9b788097992-kube-api-access-8ztxd\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:29:06.872257 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.872222    2567 generic.go:356] "Generic (PLEG): container finished" podID="4d0064db-dd09-4fe8-ad21-f9b788097992" containerID="6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219" exitCode=0
Jul 06 01:29:06.872664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.872270    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"
Jul 06 01:29:06.872664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.872305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" event={"ID":"4d0064db-dd09-4fe8-ad21-f9b788097992","Type":"ContainerDied","Data":"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"}
Jul 06 01:29:06.872664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.872352    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5" event={"ID":"4d0064db-dd09-4fe8-ad21-f9b788097992","Type":"ContainerDied","Data":"3e7a53412c7b1f5aabe6f3bbdc6673b78fa5f3bd6f26edf21abf2597ae3dae67"}
Jul 06 01:29:06.872664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.872372    2567 scope.go:117] "RemoveContainer" containerID="6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"
Jul 06 01:29:06.880621 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.880603    2567 scope.go:117] "RemoveContainer" containerID="6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"
Jul 06 01:29:06.880868 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:29:06.880847    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219\": container with ID starting with 6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219 not found: ID does not exist" containerID="6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"
Jul 06 01:29:06.880961 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.880873    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219"} err="failed to get container status \"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219\": rpc error: code = NotFound desc = could not find container \"6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219\": container with ID starting with 6c32cb9014639a10207313555b32e8ac88db49a8dd1a796520e2f7c98c685219 not found: ID does not exist"
Jul 06 01:29:06.892447 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.892430    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:29:06.895253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:06.895235    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ghqp5"]
Jul 06 01:29:08.655636 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:08.655603    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0064db-dd09-4fe8-ad21-f9b788097992" path="/var/lib/kubelet/pods/4d0064db-dd09-4fe8-ad21-f9b788097992/volumes"
Jul 06 01:29:15.630464 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.630428    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:29:15.630937 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.630918    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0064db-dd09-4fe8-ad21-f9b788097992" containerName="manager"
Jul 06 01:29:15.631017 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.630941    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0064db-dd09-4fe8-ad21-f9b788097992" containerName="manager"
Jul 06 01:29:15.631070 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.631016    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0064db-dd09-4fe8-ad21-f9b788097992" containerName="manager"
Jul 06 01:29:15.686094 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.686057    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:29:15.686094 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.686097    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:29:15.686356 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.686184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.688365 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.688343    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 01:29:15.763440 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.763392    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhg8\" (UniqueName: \"kubernetes.io/projected/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-kube-api-access-nxhg8\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.763440 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.763439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-config-file\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.864512 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.864477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhg8\" (UniqueName: \"kubernetes.io/projected/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-kube-api-access-nxhg8\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.864512 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.864517    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-config-file\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.865154 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.865104    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-config-file\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.872158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.872135    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhg8\" (UniqueName: \"kubernetes.io/projected/c61e84f1-a5da-461c-87b8-e4b4dd5d8d76-kube-api-access-nxhg8\") pod \"limitador-limitador-54d6d988d5-k5mcl\" (UID: \"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:15.996660 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:15.996582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:16.131490 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.131461    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:29:16.134573 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:29:16.134549    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61e84f1_a5da_461c_87b8_e4b4dd5d8d76.slice/crio-9ac751926ea38df41e8f792816eef36800ce58a4782a18fb29fae8e3a5e73b7f WatchSource:0}: Error finding container 9ac751926ea38df41e8f792816eef36800ce58a4782a18fb29fae8e3a5e73b7f: Status 404 returned error can't find the container with id 9ac751926ea38df41e8f792816eef36800ce58a4782a18fb29fae8e3a5e73b7f
Jul 06 01:29:16.213606 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.213580    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:16.218057 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.218036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:16.220291 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.220275    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6qm4q\""
Jul 06 01:29:16.223038 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.223017    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:16.269509 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.269447    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsg6l\" (UniqueName: \"kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l\") pod \"authorino-75d556bd77-kcm9h\" (UID: \"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6\") " pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:16.370652 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.370621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsg6l\" (UniqueName: \"kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l\") pod \"authorino-75d556bd77-kcm9h\" (UID: \"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6\") " pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:16.377917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.377886    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsg6l\" (UniqueName: \"kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l\") pod \"authorino-75d556bd77-kcm9h\" (UID: \"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6\") " pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:16.528361 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.528285    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:16.699140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.699091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:16.701897 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:29:16.701867    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ac9fa3b_64a1_4589_b93f_576dfb8cc7c6.slice/crio-755e2b37323ec7ca7d4401f8ef6e372d0d79418adea06bfaafa21d609dd60e32 WatchSource:0}: Error finding container 755e2b37323ec7ca7d4401f8ef6e372d0d79418adea06bfaafa21d609dd60e32: Status 404 returned error can't find the container with id 755e2b37323ec7ca7d4401f8ef6e372d0d79418adea06bfaafa21d609dd60e32
Jul 06 01:29:16.924170 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.924114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcm9h" event={"ID":"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6","Type":"ContainerStarted","Data":"755e2b37323ec7ca7d4401f8ef6e372d0d79418adea06bfaafa21d609dd60e32"}
Jul 06 01:29:16.925381 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:16.925354    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl" event={"ID":"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76","Type":"ContainerStarted","Data":"9ac751926ea38df41e8f792816eef36800ce58a4782a18fb29fae8e3a5e73b7f"}
Jul 06 01:29:20.953009 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:20.952970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcm9h" event={"ID":"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6","Type":"ContainerStarted","Data":"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"}
Jul 06 01:29:20.954527 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:20.954504    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl" event={"ID":"c61e84f1-a5da-461c-87b8-e4b4dd5d8d76","Type":"ContainerStarted","Data":"3c3377da8cee4b2c8f84ac02ac09ebf384fc306850974cc29cf06f9820e2728d"}
Jul 06 01:29:20.954641 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:20.954625    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:20.966619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:20.966566    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kcm9h" podStartSLOduration=1.211369321 podStartE2EDuration="4.966551718s" totalImagesPullingTime="3.755182397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:20.966462463 +0000 UTC m=+560.811611876" watchObservedRunningTime="2026-07-06 01:29:20.966551718 +0000 UTC m=+560.811701132"
Jul 06 01:29:20.983256 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:20.983204    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl" podStartSLOduration=1.668373085 podStartE2EDuration="5.983189411s" totalImagesPullingTime="4.314816326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:20.982577433 +0000 UTC m=+560.827726868" watchObservedRunningTime="2026-07-06 01:29:20.983189411 +0000 UTC m=+560.828338861"
Jul 06 01:29:31.961180 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:31.961154    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"
Jul 06 01:29:50.175316 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.175286    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:50.175734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.175468    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kcm9h" podUID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" containerName="authorino" containerID="cri-o://1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20" gracePeriod=30
Jul 06 01:29:50.478035 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.478006    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:50.548419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.548390    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsg6l\" (UniqueName: \"kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l\") pod \"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6\" (UID: \"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6\") "
Jul 06 01:29:50.550592 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.550558    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l" (OuterVolumeSpecName: "kube-api-access-tsg6l") pod "2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" (UID: "2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6"). InnerVolumeSpecName "kube-api-access-tsg6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:29:50.649507 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:50.649474    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsg6l\" (UniqueName: \"kubernetes.io/projected/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6-kube-api-access-tsg6l\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:29:51.087812 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.087780    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" containerID="1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20" exitCode=0
Jul 06 01:29:51.087992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.087823    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcm9h"
Jul 06 01:29:51.087992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.087865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcm9h" event={"ID":"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6","Type":"ContainerDied","Data":"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"}
Jul 06 01:29:51.087992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.087909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcm9h" event={"ID":"2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6","Type":"ContainerDied","Data":"755e2b37323ec7ca7d4401f8ef6e372d0d79418adea06bfaafa21d609dd60e32"}
Jul 06 01:29:51.087992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.087930    2567 scope.go:117] "RemoveContainer" containerID="1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"
Jul 06 01:29:51.096263 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.096244    2567 scope.go:117] "RemoveContainer" containerID="1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"
Jul 06 01:29:51.096515 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:29:51.096499    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20\": container with ID starting with 1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20 not found: ID does not exist" containerID="1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"
Jul 06 01:29:51.096563 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.096527    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20"} err="failed to get container status \"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20\": rpc error: code = NotFound desc = could not find container \"1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20\": container with ID starting with 1d1ade6f17305c1631edcfb5c398210910760e6f9f80438b9d0cd90512017f20 not found: ID does not exist"
Jul 06 01:29:51.102898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.102879    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:51.106960 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:51.106940    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcm9h"]
Jul 06 01:29:52.656003 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:29:52.655960    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" path="/var/lib/kubelet/pods/2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6/volumes"
Jul 06 01:30:00.590798 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:30:00.590771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:30:00.592722 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:30:00.592703    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:30:29.634459 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:30:29.634377    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:30:48.988943 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:30:48.988912    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:09.464144 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:09.464095    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:14.153764 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:14.153730    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:16.561618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.561583    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"]
Jul 06 01:31:16.562090 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.562074    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" containerName="authorino"
Jul 06 01:31:16.562156 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.562094    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" containerName="authorino"
Jul 06 01:31:16.562232 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.562219    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac9fa3b-64a1-4589-b93f-576dfb8cc7c6" containerName="authorino"
Jul 06 01:31:16.565605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.565584    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.568830 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.568812    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 06 01:31:16.568944 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.568852    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 01:31:16.568944 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.568854    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f2mpd\""
Jul 06 01:31:16.568944 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.568852    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 01:31:16.575820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.575796    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"]
Jul 06 01:31:16.642206 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.642358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.642358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642237    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.642358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkppc\" (UniqueName: \"kubernetes.io/projected/67e24451-d74e-4487-ae42-243fe9dd2c73-kube-api-access-nkppc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.642462 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e24451-d74e-4487-ae42-243fe9dd2c73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.642462 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.642429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.742822 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.742788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743003 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.742831    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkppc\" (UniqueName: \"kubernetes.io/projected/67e24451-d74e-4487-ae42-243fe9dd2c73-kube-api-access-nkppc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743003 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.742859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e24451-d74e-4487-ae42-243fe9dd2c73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743003 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.742905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743003 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.742980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743257 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.743013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743347 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.743322    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.743372    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.743455 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.743408    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.745343 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.745322    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e24451-d74e-4487-ae42-243fe9dd2c73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.745530 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.745513    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e24451-d74e-4487-ae42-243fe9dd2c73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.750738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.750717    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkppc\" (UniqueName: \"kubernetes.io/projected/67e24451-d74e-4487-ae42-243fe9dd2c73-kube-api-access-nkppc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv\" (UID: \"67e24451-d74e-4487-ae42-243fe9dd2c73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:16.877920 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:16.877897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:17.011904 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:17.011866    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"]
Jul 06 01:31:17.013263 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:31:17.013232    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e24451_d74e_4487_ae42_243fe9dd2c73.slice/crio-37069679c0bcba43ce913adddf9bcceb29433bc50f730aa4e5b503984321d416 WatchSource:0}: Error finding container 37069679c0bcba43ce913adddf9bcceb29433bc50f730aa4e5b503984321d416: Status 404 returned error can't find the container with id 37069679c0bcba43ce913adddf9bcceb29433bc50f730aa4e5b503984321d416
Jul 06 01:31:17.014686 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:17.014664    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 01:31:17.476022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:17.475989    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv" event={"ID":"67e24451-d74e-4487-ae42-243fe9dd2c73","Type":"ContainerStarted","Data":"37069679c0bcba43ce913adddf9bcceb29433bc50f730aa4e5b503984321d416"}
Jul 06 01:31:18.164944 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:18.164910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:18.481774 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:18.481739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv" event={"ID":"67e24451-d74e-4487-ae42-243fe9dd2c73","Type":"ContainerStarted","Data":"6c9d9d36e4d992372825bbec3fb85ff43d93a8aa4d4849c04f53f085e1d3a3b5"}
Jul 06 01:31:18.497778 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:18.497726    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv" podStartSLOduration=1.485076194 podStartE2EDuration="2.497711402s" totalImagesPullingTime="1.012635208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:18.496622927 +0000 UTC m=+678.341772358" watchObservedRunningTime="2026-07-06 01:31:18.497711402 +0000 UTC m=+678.342860814"
Jul 06 01:31:26.878131 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:26.878095    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:26.878594 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:26.878173    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:26.890807 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:26.890785    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:27.534219 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:27.534194    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv"
Jul 06 01:31:31.859796 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:31.859763    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:42.562886 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.562854    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"]
Jul 06 01:31:42.566240 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.566221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.568581 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.568561    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 06 01:31:42.576083 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.576064    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"]
Jul 06 01:31:42.649946 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.649910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.650084 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.649973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.650084 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.650022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lcs\" (UniqueName: \"kubernetes.io/projected/c526d51f-6859-4b53-b04d-c1ff3019ab42-kube-api-access-67lcs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.650222 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.650083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c526d51f-6859-4b53-b04d-c1ff3019ab42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.650222 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.650108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.650308 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.650211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751391    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lcs\" (UniqueName: \"kubernetes.io/projected/c526d51f-6859-4b53-b04d-c1ff3019ab42-kube-api-access-67lcs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751591 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c526d51f-6859-4b53-b04d-c1ff3019ab42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751591 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751591 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751582    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751802    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751909 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.751969 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.751954    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.753978 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.753956    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c526d51f-6859-4b53-b04d-c1ff3019ab42-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.754159 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.754141    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c526d51f-6859-4b53-b04d-c1ff3019ab42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.758814 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.758797    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lcs\" (UniqueName: \"kubernetes.io/projected/c526d51f-6859-4b53-b04d-c1ff3019ab42-kube-api-access-67lcs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6\" (UID: \"c526d51f-6859-4b53-b04d-c1ff3019ab42\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:42.875654 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:42.875622    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:43.005297 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:43.005269    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"]
Jul 06 01:31:43.008198 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:31:43.008169    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc526d51f_6859_4b53_b04d_c1ff3019ab42.slice/crio-8b57c6ed306740269dae36650184c34d2e8d4210cab4ce1790b823530d53ee87 WatchSource:0}: Error finding container 8b57c6ed306740269dae36650184c34d2e8d4210cab4ce1790b823530d53ee87: Status 404 returned error can't find the container with id 8b57c6ed306740269dae36650184c34d2e8d4210cab4ce1790b823530d53ee87
Jul 06 01:31:43.595660 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:43.595566    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6" event={"ID":"c526d51f-6859-4b53-b04d-c1ff3019ab42","Type":"ContainerStarted","Data":"26c61572d3790687b3b3697d161907c2c1e068c131b25ef7b95aa912203754ab"}
Jul 06 01:31:43.595660 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:43.595604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6" event={"ID":"c526d51f-6859-4b53-b04d-c1ff3019ab42","Type":"ContainerStarted","Data":"8b57c6ed306740269dae36650184c34d2e8d4210cab4ce1790b823530d53ee87"}
Jul 06 01:31:43.612937 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:43.612891    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6" podStartSLOduration=1.446659247 podStartE2EDuration="1.612876893s" totalImagesPullingTime="166.217646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:43.611374193 +0000 UTC m=+703.456523606" watchObservedRunningTime="2026-07-06 01:31:43.612876893 +0000 UTC m=+703.458026304"
Jul 06 01:31:44.963403 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:44.963369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:31:52.876415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:52.876312    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:52.876415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:52.876371    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:52.888751 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:52.888727    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:31:53.655250 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:31:53.655223    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6"
Jul 06 01:32:15.460598 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.460566    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ff6f8b7f4-6xj7v"]
Jul 06 01:32:15.463778 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.463761    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.466981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.466956    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6qm4q\""
Jul 06 01:32:15.467102 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.467000    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 01:32:15.471140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.471104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ff6f8b7f4-6xj7v"]
Jul 06 01:32:15.531264 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.531233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-tls-cert\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.531436 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.531342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgtg8\" (UniqueName: \"kubernetes.io/projected/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-kube-api-access-rgtg8\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.632434 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.632405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-tls-cert\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.632600 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.632465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgtg8\" (UniqueName: \"kubernetes.io/projected/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-kube-api-access-rgtg8\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.635005 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.634973    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-tls-cert\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.640364 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.640342    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgtg8\" (UniqueName: \"kubernetes.io/projected/cbb15602-250e-4da9-9ab1-bcb7a34f60f4-kube-api-access-rgtg8\") pod \"authorino-5ff6f8b7f4-6xj7v\" (UID: \"cbb15602-250e-4da9-9ab1-bcb7a34f60f4\") " pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.774083 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.774012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v"
Jul 06 01:32:15.905824 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:15.905748    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ff6f8b7f4-6xj7v"]
Jul 06 01:32:15.908296 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:32:15.908258    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbb15602_250e_4da9_9ab1_bcb7a34f60f4.slice/crio-1d54738a160eed065421cbdb4eebc4b7eb3d30601db70f3b9b37bea9f09476a2 WatchSource:0}: Error finding container 1d54738a160eed065421cbdb4eebc4b7eb3d30601db70f3b9b37bea9f09476a2: Status 404 returned error can't find the container with id 1d54738a160eed065421cbdb4eebc4b7eb3d30601db70f3b9b37bea9f09476a2
Jul 06 01:32:16.750897 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:16.750863    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v" event={"ID":"cbb15602-250e-4da9-9ab1-bcb7a34f60f4","Type":"ContainerStarted","Data":"af86b7eab2dd58d294d39cb04628ee9acbecfa3ee74c75504bb93860bd493638"}
Jul 06 01:32:16.751326 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:16.750904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v" event={"ID":"cbb15602-250e-4da9-9ab1-bcb7a34f60f4","Type":"ContainerStarted","Data":"1d54738a160eed065421cbdb4eebc4b7eb3d30601db70f3b9b37bea9f09476a2"}
Jul 06 01:32:16.774221 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:16.774161    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ff6f8b7f4-6xj7v" podStartSLOduration=1.40963824 podStartE2EDuration="1.774140705s" totalImagesPullingTime="364.502465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:32:16.771611411 +0000 UTC m=+736.616760830" watchObservedRunningTime="2026-07-06 01:32:16.774140705 +0000 UTC m=+736.619290109"
Jul 06 01:32:45.668650 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:45.668615    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:32:55.458853 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:32:55.458820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:33:07.060023 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:33:07.059992    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:33:15.655984 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:33:15.655947    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:33:26.263461 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:33:26.263375    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:33:51.257863 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:33:51.257831    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:34:03.364625 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:34:03.364584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:34:37.060432 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:34:37.060393    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:34:46.859842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:34:46.859800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:34:55.166974 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:34:55.166895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:35:00.619272 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:00.619223    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:35:00.619741 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:00.619442    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:35:03.969056 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:03.969023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:35:14.559136 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:14.559089    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:35:21.067647 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:21.067615    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:35:37.764281 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:37.764246    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:35:47.178338 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:35:47.178304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:36:09.564984 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:36:09.564952    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:36:20.956217 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:36:20.956186    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:36:44.958719 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:36:44.958638    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:36:55.356903 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:36:55.356867    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:37:10.459367 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:37:10.459331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:37:22.361671 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:37:22.361629    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:37:36.665033 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:37:36.665005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:38:15.406747 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:15.406704    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:38:15.407370 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:15.406994    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" podUID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" containerName="manager" containerID="cri-o://512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02" gracePeriod=10
Jul 06 01:38:18.356944 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.356922    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:38:18.409339 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.409304    2567 generic.go:356] "Generic (PLEG): container finished" podID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" containerID="512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02" exitCode=0
Jul 06 01:38:18.409339 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.409342    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"
Jul 06 01:38:18.409576 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.409388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" event={"ID":"46c0f473-a7b3-4658-8ff2-1b196941d6a2","Type":"ContainerDied","Data":"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"}
Jul 06 01:38:18.409576 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.409436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl" event={"ID":"46c0f473-a7b3-4658-8ff2-1b196941d6a2","Type":"ContainerDied","Data":"c89861493ac16a6f8db488e964215dbafccc6c73258ce6e9d8cbd9495910972e"}
Jul 06 01:38:18.409576 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.409463    2567 scope.go:117] "RemoveContainer" containerID="512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"
Jul 06 01:38:18.419350 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.419330    2567 scope.go:117] "RemoveContainer" containerID="512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"
Jul 06 01:38:18.419638 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:38:18.419616    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02\": container with ID starting with 512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02 not found: ID does not exist" containerID="512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"
Jul 06 01:38:18.419721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.419648    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02"} err="failed to get container status \"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02\": rpc error: code = NotFound desc = could not find container \"512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02\": container with ID starting with 512e07f55adbb985ee7e1157ac18195f8c516843e3c6b63f53343cc3f496fb02 not found: ID does not exist"
Jul 06 01:38:18.446952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.446932    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8slw8\" (UniqueName: \"kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8\") pod \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") "
Jul 06 01:38:18.447065 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.446970    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume\") pod \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\" (UID: \"46c0f473-a7b3-4658-8ff2-1b196941d6a2\") "
Jul 06 01:38:18.447380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.447356    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "46c0f473-a7b3-4658-8ff2-1b196941d6a2" (UID: "46c0f473-a7b3-4658-8ff2-1b196941d6a2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:38:18.449031 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.449013    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8" (OuterVolumeSpecName: "kube-api-access-8slw8") pod "46c0f473-a7b3-4658-8ff2-1b196941d6a2" (UID: "46c0f473-a7b3-4658-8ff2-1b196941d6a2"). InnerVolumeSpecName "kube-api-access-8slw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:38:18.547980 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.547908    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46c0f473-a7b3-4658-8ff2-1b196941d6a2-extensions-socket-volume\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:38:18.547980 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.547937    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8slw8\" (UniqueName: \"kubernetes.io/projected/46c0f473-a7b3-4658-8ff2-1b196941d6a2-kube-api-access-8slw8\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:38:18.725112 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.725080    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:38:18.728971 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:18.728949    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kh7rl"]
Jul 06 01:38:20.661295 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:38:20.661261    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" path="/var/lib/kubelet/pods/46c0f473-a7b3-4658-8ff2-1b196941d6a2/volumes"
Jul 06 01:39:36.463925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:39:36.463886    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:39:45.258863 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:39:45.258827    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:39:57.563850 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:39:57.563814    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:40:00.648514 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:00.648490    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:40:00.652198 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:00.652175    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:40:09.367923 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:09.367886    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:40:20.646997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:20.646962    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:40:31.552462 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:31.552423    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:40:40.154028 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:40.153990    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:40:51.160147 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:40:51.160103    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:41:01.756561 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:41:01.756483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:41:11.254388 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:41:11.254354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:41:20.955293 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:41:20.955254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:41:45.754792 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:41:45.754758    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:41:53.453192 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:41:53.453155    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:42:08.453555 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:42:08.453514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:42:17.256367 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:42:17.256332    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:42:33.860906 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:42:33.860830    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:42:42.956740 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:42:42.956705    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:42:52.157397 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:42:52.157360    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:01.858325 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:01.858291    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:10.456735 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:10.456702    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:19.461092 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:19.461054    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:32.556793 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:32.556761    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:41.753755 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:41.753720    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:43:52.656572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:43:52.656485    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:02.449757 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:02.449723    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:14.561048 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:14.561014    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:22.460566 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:22.460530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:32.353295 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:32.353254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:43.856274 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:43.856240    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:44:52.657014 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:44:52.656984    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:00.674236 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:00.674207    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:45:00.678773 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:00.678752    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:45:05.158832 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:05.158796    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:14.656830 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:14.656777    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:22.954683 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:22.954605    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:31.859694 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:31.859662    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:33.668663 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.668624    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:33.669167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.669098    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" containerName="manager"
Jul 06 01:45:33.669167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.669139    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" containerName="manager"
Jul 06 01:45:33.669286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.669263    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="46c0f473-a7b3-4658-8ff2-1b196941d6a2" containerName="manager"
Jul 06 01:45:33.672190 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.672171    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.674290 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.674270    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-836a1b7b-serving-cert\""
Jul 06 01:45:33.674378 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.674358    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:45:33.681960 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.681940    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:33.773180 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.773153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzh9f\" (UniqueName: \"kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.773343 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.773209    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.773343 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.773268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.874669 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.874640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.874810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.874686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzh9f\" (UniqueName: \"kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.874810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.874776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.874933 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:45:33.874917    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-836a1b7b-serving-cert: secret "maas-api-e2e-ait-836a1b7b-serving-cert" not found
Jul 06 01:45:33.875001 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:45:33.874991    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls podName:0d4bad05-5faf-4a8d-9551-21d7d4444022 nodeName:}" failed. No retries permitted until 2026-07-06 01:45:34.374969241 +0000 UTC m=+1534.220118631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls") pod "maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" (UID: "0d4bad05-5faf-4a8d-9551-21d7d4444022") : secret "maas-api-e2e-ait-836a1b7b-serving-cert" not found
Jul 06 01:45:33.875432 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.875413    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:33.883753 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:33.883730    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzh9f\" (UniqueName: \"kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:34.379830 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:34.379795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:34.382349 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:34.382325    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") pod \"maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") " pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:34.582541 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:34.582509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:34.719085 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:34.719063    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:34.721571 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:45:34.721542    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4bad05_5faf_4a8d_9551_21d7d4444022.slice/crio-39adc67100f02ac09ddeb51533e243a654aa703e70b48aca63bb1abecbe49734 WatchSource:0}: Error finding container 39adc67100f02ac09ddeb51533e243a654aa703e70b48aca63bb1abecbe49734: Status 404 returned error can't find the container with id 39adc67100f02ac09ddeb51533e243a654aa703e70b48aca63bb1abecbe49734
Jul 06 01:45:34.723170 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:34.723147    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 01:45:35.370187 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:35.370138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" event={"ID":"0d4bad05-5faf-4a8d-9551-21d7d4444022","Type":"ContainerStarted","Data":"39adc67100f02ac09ddeb51533e243a654aa703e70b48aca63bb1abecbe49734"}
Jul 06 01:45:37.382602 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:37.382567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" event={"ID":"0d4bad05-5faf-4a8d-9551-21d7d4444022","Type":"ContainerStarted","Data":"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"}
Jul 06 01:45:37.382957 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:37.382647    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:37.399091 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:37.399046    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" podStartSLOduration=2.083796391 podStartE2EDuration="4.399030277s" totalImagesPullingTime="2.315233886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:45:37.398291049 +0000 UTC m=+1537.243440461" watchObservedRunningTime="2026-07-06 01:45:37.399030277 +0000 UTC m=+1537.244179717"
Jul 06 01:45:38.917912 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:38.917872    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:39.393184 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.393149    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" podUID="0d4bad05-5faf-4a8d-9551-21d7d4444022" containerName="maas-api" containerID="cri-o://c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4" gracePeriod=30
Jul 06 01:45:39.635167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.635143    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:39.730469 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.730394    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzh9f\" (UniqueName: \"kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f\") pod \"0d4bad05-5faf-4a8d-9551-21d7d4444022\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") "
Jul 06 01:45:39.730469 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.730425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca\") pod \"0d4bad05-5faf-4a8d-9551-21d7d4444022\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") "
Jul 06 01:45:39.730654 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.730501    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") pod \"0d4bad05-5faf-4a8d-9551-21d7d4444022\" (UID: \"0d4bad05-5faf-4a8d-9551-21d7d4444022\") "
Jul 06 01:45:39.730835 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.730808    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d4bad05-5faf-4a8d-9551-21d7d4444022" (UID: "0d4bad05-5faf-4a8d-9551-21d7d4444022"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:45:39.732765 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.732739    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d4bad05-5faf-4a8d-9551-21d7d4444022" (UID: "0d4bad05-5faf-4a8d-9551-21d7d4444022"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:45:39.732864 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.732789    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f" (OuterVolumeSpecName: "kube-api-access-zzh9f") pod "0d4bad05-5faf-4a8d-9551-21d7d4444022" (UID: "0d4bad05-5faf-4a8d-9551-21d7d4444022"). InnerVolumeSpecName "kube-api-access-zzh9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:45:39.831539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.831508    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4bad05-5faf-4a8d-9551-21d7d4444022-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:39.831539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.831533    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzh9f\" (UniqueName: \"kubernetes.io/projected/0d4bad05-5faf-4a8d-9551-21d7d4444022-kube-api-access-zzh9f\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:39.831539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:39.831543    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4bad05-5faf-4a8d-9551-21d7d4444022-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:40.399894 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.399855    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d4bad05-5faf-4a8d-9551-21d7d4444022" containerID="c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4" exitCode=0
Jul 06 01:45:40.400414 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.399900    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"
Jul 06 01:45:40.400414 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.399935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" event={"ID":"0d4bad05-5faf-4a8d-9551-21d7d4444022","Type":"ContainerDied","Data":"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"}
Jul 06 01:45:40.400414 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.399992    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n" event={"ID":"0d4bad05-5faf-4a8d-9551-21d7d4444022","Type":"ContainerDied","Data":"39adc67100f02ac09ddeb51533e243a654aa703e70b48aca63bb1abecbe49734"}
Jul 06 01:45:40.400414 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.400023    2567 scope.go:117] "RemoveContainer" containerID="c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"
Jul 06 01:45:40.410687 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.410641    2567 scope.go:117] "RemoveContainer" containerID="c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"
Jul 06 01:45:40.410946 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:45:40.410919    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4\": container with ID starting with c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4 not found: ID does not exist" containerID="c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"
Jul 06 01:45:40.410999 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.410959    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4"} err="failed to get container status \"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4\": rpc error: code = NotFound desc = could not find container \"c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4\": container with ID starting with c5e7f2121c5798ca933b9ca41f62b2c0c68f846155d731ebd272835c42996ea4 not found: ID does not exist"
Jul 06 01:45:40.420474 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.420452    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:40.424428 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.424380    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-836a1b7b-6d4cf5c9d4-vf59n"]
Jul 06 01:45:40.657786 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:40.657707    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d4bad05-5faf-4a8d-9551-21d7d4444022" path="/var/lib/kubelet/pods/0d4bad05-5faf-4a8d-9551-21d7d4444022/volumes"
Jul 06 01:45:45.844743 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.844717    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:45.845137 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.845034    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4bad05-5faf-4a8d-9551-21d7d4444022" containerName="maas-api"
Jul 06 01:45:45.845137 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.845045    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4bad05-5faf-4a8d-9551-21d7d4444022" containerName="maas-api"
Jul 06 01:45:45.845137 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.845112    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4bad05-5faf-4a8d-9551-21d7d4444022" containerName="maas-api"
Jul 06 01:45:45.849481 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.849457    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:45.851719 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.851693    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-22d9eeec-serving-cert\""
Jul 06 01:45:45.851826 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.851803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:45:45.857747 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.857722    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:45.982760 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.982727    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:45.982925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.982771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:45.982925 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:45.982814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6jhd\" (UniqueName: \"kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.083453 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.083420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6jhd\" (UniqueName: \"kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.083595 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.083507    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.083595 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.083540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.084105 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.084085    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.086106 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.086086    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.093377 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.093354    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6jhd\" (UniqueName: \"kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd\") pod \"maas-api-e2e-ait-22d9eeec-974577b69-6l7td\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") " pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.160691 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.160665    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:46.297104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.297082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:46.299507 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:45:46.299479    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314383b0_b302_4117_b594_de73258ae16e.slice/crio-f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5 WatchSource:0}: Error finding container f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5: Status 404 returned error can't find the container with id f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5
Jul 06 01:45:46.429791 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:46.429705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" event={"ID":"314383b0-b302-4117-b594-de73258ae16e","Type":"ContainerStarted","Data":"f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5"}
Jul 06 01:45:47.436244 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:47.436212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" event={"ID":"314383b0-b302-4117-b594-de73258ae16e","Type":"ContainerStarted","Data":"ef452d06859f4e295c33a39dae2db98cd51c9d418b5d539b04cb34f073f72b66"}
Jul 06 01:45:47.436644 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:47.436289    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:47.467836 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:47.467786    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" podStartSLOduration=2.162993455 podStartE2EDuration="2.467771629s" totalImagesPullingTime="304.778174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:45:47.466691766 +0000 UTC m=+1547.311841198" watchObservedRunningTime="2026-07-06 01:45:47.467771629 +0000 UTC m=+1547.312921041"
Jul 06 01:45:48.154264 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:48.154230    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:45:51.226669 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.226534    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:51.251555 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.226752    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" podUID="314383b0-b302-4117-b594-de73258ae16e" containerName="maas-api" containerID="cri-o://ef452d06859f4e295c33a39dae2db98cd51c9d418b5d539b04cb34f073f72b66" gracePeriod=30
Jul 06 01:45:51.251555 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.232841    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:51.456765 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.456742    2567 generic.go:356] "Generic (PLEG): container finished" podID="314383b0-b302-4117-b594-de73258ae16e" containerID="ef452d06859f4e295c33a39dae2db98cd51c9d418b5d539b04cb34f073f72b66" exitCode=0
Jul 06 01:45:51.456881 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.456825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" event={"ID":"314383b0-b302-4117-b594-de73258ae16e","Type":"ContainerDied","Data":"ef452d06859f4e295c33a39dae2db98cd51c9d418b5d539b04cb34f073f72b66"}
Jul 06 01:45:51.456932 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.456884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td" event={"ID":"314383b0-b302-4117-b594-de73258ae16e","Type":"ContainerDied","Data":"f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5"}
Jul 06 01:45:51.456932 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.456910    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f161f5375743e4b4e30359ea17e90a9e58341bc836b4db73a0a7673265bfcca5"
Jul 06 01:45:51.469434 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.469416    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:51.630177 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.630152    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls\") pod \"314383b0-b302-4117-b594-de73258ae16e\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") "
Jul 06 01:45:51.630350 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.630197    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca\") pod \"314383b0-b302-4117-b594-de73258ae16e\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") "
Jul 06 01:45:51.630350 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.630216    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6jhd\" (UniqueName: \"kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd\") pod \"314383b0-b302-4117-b594-de73258ae16e\" (UID: \"314383b0-b302-4117-b594-de73258ae16e\") "
Jul 06 01:45:51.630748 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.630715    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "314383b0-b302-4117-b594-de73258ae16e" (UID: "314383b0-b302-4117-b594-de73258ae16e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:45:51.632580 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.632556    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "314383b0-b302-4117-b594-de73258ae16e" (UID: "314383b0-b302-4117-b594-de73258ae16e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:45:51.632848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.632817    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd" (OuterVolumeSpecName: "kube-api-access-g6jhd") pod "314383b0-b302-4117-b594-de73258ae16e" (UID: "314383b0-b302-4117-b594-de73258ae16e"). InnerVolumeSpecName "kube-api-access-g6jhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:45:51.731574 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.731546    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/314383b0-b302-4117-b594-de73258ae16e-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:51.731574 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.731569    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/314383b0-b302-4117-b594-de73258ae16e-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:51.731574 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:51.731578    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6jhd\" (UniqueName: \"kubernetes.io/projected/314383b0-b302-4117-b594-de73258ae16e-kube-api-access-g6jhd\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:45:52.461165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:52.461134    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"
Jul 06 01:45:52.481404 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:52.481375    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:52.485440 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:52.485414    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-22d9eeec-974577b69-6l7td"]
Jul 06 01:45:52.656187 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:52.656148    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314383b0-b302-4117-b594-de73258ae16e" path="/var/lib/kubelet/pods/314383b0-b302-4117-b594-de73258ae16e/volumes"
Jul 06 01:45:57.655376 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:45:57.655342    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:46:04.410900 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.410870    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:04.411445 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.411263    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314383b0-b302-4117-b594-de73258ae16e" containerName="maas-api"
Jul 06 01:46:04.411445 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.411276    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="314383b0-b302-4117-b594-de73258ae16e" containerName="maas-api"
Jul 06 01:46:04.411445 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.411348    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="314383b0-b302-4117-b594-de73258ae16e" containerName="maas-api"
Jul 06 01:46:04.416360 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.416339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.418888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.418868    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-f42a3931-serving-cert\""
Jul 06 01:46:04.418992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.418895    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:46:04.423914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.423890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:04.429794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.429766    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.429879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.429811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.429923 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.429906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfvwr\" (UniqueName: \"kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.530435 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.530408    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfvwr\" (UniqueName: \"kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.530630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.530453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.530630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.530472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.531165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.531141    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.533254 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.533232    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.543961 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.543936    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfvwr\" (UniqueName: \"kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr\") pod \"maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") " pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.728054 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.727981    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:04.887156 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:04.887134    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:04.889962 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:46:04.889933    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaebcc29_5103_4636_b685_8bfa1992bfbb.slice/crio-e98310b61156867d03b092374750ee66f72e2fd03d653430ab7638d91a9295ea WatchSource:0}: Error finding container e98310b61156867d03b092374750ee66f72e2fd03d653430ab7638d91a9295ea: Status 404 returned error can't find the container with id e98310b61156867d03b092374750ee66f72e2fd03d653430ab7638d91a9295ea
Jul 06 01:46:05.521486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:05.521397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" event={"ID":"baebcc29-5103-4636-b685-8bfa1992bfbb","Type":"ContainerStarted","Data":"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"}
Jul 06 01:46:05.521486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:05.521454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" event={"ID":"baebcc29-5103-4636-b685-8bfa1992bfbb","Type":"ContainerStarted","Data":"e98310b61156867d03b092374750ee66f72e2fd03d653430ab7638d91a9295ea"}
Jul 06 01:46:05.521486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:05.521476    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:05.539470 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:05.539403    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" podStartSLOduration=1.237128179 podStartE2EDuration="1.539382825s" totalImagesPullingTime="302.254646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:46:05.537859985 +0000 UTC m=+1565.383009420" watchObservedRunningTime="2026-07-06 01:46:05.539382825 +0000 UTC m=+1565.384532240"
Jul 06 01:46:09.027025 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.026995    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:09.027524 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.027235    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" podUID="baebcc29-5103-4636-b685-8bfa1992bfbb" containerName="maas-api" containerID="cri-o://421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899" gracePeriod=30
Jul 06 01:46:09.269692 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.269671    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:09.366667 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.366644    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls\") pod \"baebcc29-5103-4636-b685-8bfa1992bfbb\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") "
Jul 06 01:46:09.366889 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.366687    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca\") pod \"baebcc29-5103-4636-b685-8bfa1992bfbb\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") "
Jul 06 01:46:09.366889 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.366711    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfvwr\" (UniqueName: \"kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr\") pod \"baebcc29-5103-4636-b685-8bfa1992bfbb\" (UID: \"baebcc29-5103-4636-b685-8bfa1992bfbb\") "
Jul 06 01:46:09.367050 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.367010    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "baebcc29-5103-4636-b685-8bfa1992bfbb" (UID: "baebcc29-5103-4636-b685-8bfa1992bfbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:46:09.368922 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.368900    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr" (OuterVolumeSpecName: "kube-api-access-pfvwr") pod "baebcc29-5103-4636-b685-8bfa1992bfbb" (UID: "baebcc29-5103-4636-b685-8bfa1992bfbb"). InnerVolumeSpecName "kube-api-access-pfvwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:46:09.368993 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.368936    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "baebcc29-5103-4636-b685-8bfa1992bfbb" (UID: "baebcc29-5103-4636-b685-8bfa1992bfbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:46:09.467721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.467692    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baebcc29-5103-4636-b685-8bfa1992bfbb-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:46:09.467721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.467716    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baebcc29-5103-4636-b685-8bfa1992bfbb-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:46:09.467721 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.467725    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfvwr\" (UniqueName: \"kubernetes.io/projected/baebcc29-5103-4636-b685-8bfa1992bfbb-kube-api-access-pfvwr\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:46:09.539747 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.539719    2567 generic.go:356] "Generic (PLEG): container finished" podID="baebcc29-5103-4636-b685-8bfa1992bfbb" containerID="421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899" exitCode=0
Jul 06 01:46:09.539898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.539768    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"
Jul 06 01:46:09.539898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.539774    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" event={"ID":"baebcc29-5103-4636-b685-8bfa1992bfbb","Type":"ContainerDied","Data":"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"}
Jul 06 01:46:09.539898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.539815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h" event={"ID":"baebcc29-5103-4636-b685-8bfa1992bfbb","Type":"ContainerDied","Data":"e98310b61156867d03b092374750ee66f72e2fd03d653430ab7638d91a9295ea"}
Jul 06 01:46:09.539898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.539835    2567 scope.go:117] "RemoveContainer" containerID="421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"
Jul 06 01:46:09.548934 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.548915    2567 scope.go:117] "RemoveContainer" containerID="421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"
Jul 06 01:46:09.549240 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:46:09.549217    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899\": container with ID starting with 421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899 not found: ID does not exist" containerID="421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"
Jul 06 01:46:09.549334 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.549247    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899"} err="failed to get container status \"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899\": rpc error: code = NotFound desc = could not find container \"421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899\": container with ID starting with 421d3b288a986a21e2c7c2fd85b52ef074f7596bc4bf7f4d66e64cba1ad90899 not found: ID does not exist"
Jul 06 01:46:09.560532 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.560509    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:09.571564 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:09.571542    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-f42a3931-8497b87fd8-nj57h"]
Jul 06 01:46:10.655989 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:46:10.655948    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baebcc29-5103-4636-b685-8bfa1992bfbb" path="/var/lib/kubelet/pods/baebcc29-5103-4636-b685-8bfa1992bfbb/volumes"
Jul 06 01:47:37.949260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:37.949225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:47:46.755155 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:46.755107    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:47:53.363242 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.363212    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:47:53.363834 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.363561    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baebcc29-5103-4636-b685-8bfa1992bfbb" containerName="maas-api"
Jul 06 01:47:53.363834 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.363573    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="baebcc29-5103-4636-b685-8bfa1992bfbb" containerName="maas-api"
Jul 06 01:47:53.363834 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.363635    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="baebcc29-5103-4636-b685-8bfa1992bfbb" containerName="maas-api"
Jul 06 01:47:53.366593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.366575    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.368847 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.368829    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-17e353c4-serving-cert\""
Jul 06 01:47:53.368965 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.368911    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:47:53.375744 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.375719    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:47:53.509572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.509532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4v4r\" (UniqueName: \"kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.509757 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.509580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.509757 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.509665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.610608 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.610575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4v4r\" (UniqueName: \"kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.610608 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.610613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.610874 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.610642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.610874 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:47:53.610752    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-17e353c4-serving-cert: secret "maas-api-e2e-mt-17e353c4-serving-cert" not found
Jul 06 01:47:53.610874 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:47:53.610837    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls podName:50d8967d-0e2e-40d8-a06f-5eeb52af361a nodeName:}" failed. No retries permitted until 2026-07-06 01:47:54.11081589 +0000 UTC m=+1673.955965281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls") pod "maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" (UID: "50d8967d-0e2e-40d8-a06f-5eeb52af361a") : secret "maas-api-e2e-mt-17e353c4-serving-cert" not found
Jul 06 01:47:53.611282 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.611265    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:53.618611 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:53.618587    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4v4r\" (UniqueName: \"kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:54.114966 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:54.114931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:54.117453 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:54.117432    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") pod \"maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") " pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:54.279600 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:54.279562    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:54.417322 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:54.417295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:47:54.419743 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:47:54.419708    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d8967d_0e2e_40d8_a06f_5eeb52af361a.slice/crio-3f809b5e747d361c7d3252707acbfdf6ab2726c3f94fb7517eac29ab5ffeba95 WatchSource:0}: Error finding container 3f809b5e747d361c7d3252707acbfdf6ab2726c3f94fb7517eac29ab5ffeba95: Status 404 returned error can't find the container with id 3f809b5e747d361c7d3252707acbfdf6ab2726c3f94fb7517eac29ab5ffeba95
Jul 06 01:47:55.025863 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:55.025827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" event={"ID":"50d8967d-0e2e-40d8-a06f-5eeb52af361a","Type":"ContainerStarted","Data":"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"}
Jul 06 01:47:55.026024 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:55.025879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" event={"ID":"50d8967d-0e2e-40d8-a06f-5eeb52af361a","Type":"ContainerStarted","Data":"3f809b5e747d361c7d3252707acbfdf6ab2726c3f94fb7517eac29ab5ffeba95"}
Jul 06 01:47:55.026024 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:55.025944    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:47:55.043647 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:47:55.043600    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" podStartSLOduration=1.615410814 podStartE2EDuration="2.043585727s" totalImagesPullingTime="428.174913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:47:55.041972036 +0000 UTC m=+1674.887121459" watchObservedRunningTime="2026-07-06 01:47:55.043585727 +0000 UTC m=+1674.888735138"
Jul 06 01:48:00.064649 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:00.064614    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:48:01.036395 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:01.036371    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:48:05.554483 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.554451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:48:05.554914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.554706    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" podUID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" containerName="maas-api" containerID="cri-o://b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac" gracePeriod=30
Jul 06 01:48:05.804647 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.804589    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:48:05.816483 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.816455    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca\") pod \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") "
Jul 06 01:48:05.816586 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.816492    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") pod \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") "
Jul 06 01:48:05.816586 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.816538    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4v4r\" (UniqueName: \"kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r\") pod \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\" (UID: \"50d8967d-0e2e-40d8-a06f-5eeb52af361a\") "
Jul 06 01:48:05.816822 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.816798    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50d8967d-0e2e-40d8-a06f-5eeb52af361a" (UID: "50d8967d-0e2e-40d8-a06f-5eeb52af361a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:48:05.818675 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.818653    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50d8967d-0e2e-40d8-a06f-5eeb52af361a" (UID: "50d8967d-0e2e-40d8-a06f-5eeb52af361a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:48:05.818750 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.818669    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r" (OuterVolumeSpecName: "kube-api-access-q4v4r") pod "50d8967d-0e2e-40d8-a06f-5eeb52af361a" (UID: "50d8967d-0e2e-40d8-a06f-5eeb52af361a"). InnerVolumeSpecName "kube-api-access-q4v4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:48:05.917330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.917299    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d8967d-0e2e-40d8-a06f-5eeb52af361a-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:48:05.917330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.917324    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d8967d-0e2e-40d8-a06f-5eeb52af361a-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:48:05.917330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:05.917333    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4v4r\" (UniqueName: \"kubernetes.io/projected/50d8967d-0e2e-40d8-a06f-5eeb52af361a-kube-api-access-q4v4r\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:48:06.078409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.078325    2567 generic.go:356] "Generic (PLEG): container finished" podID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" containerID="b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac" exitCode=0
Jul 06 01:48:06.078409 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.078379    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"
Jul 06 01:48:06.078618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.078385    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" event={"ID":"50d8967d-0e2e-40d8-a06f-5eeb52af361a","Type":"ContainerDied","Data":"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"}
Jul 06 01:48:06.078618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.078499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58" event={"ID":"50d8967d-0e2e-40d8-a06f-5eeb52af361a","Type":"ContainerDied","Data":"3f809b5e747d361c7d3252707acbfdf6ab2726c3f94fb7517eac29ab5ffeba95"}
Jul 06 01:48:06.078618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.078519    2567 scope.go:117] "RemoveContainer" containerID="b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"
Jul 06 01:48:06.087551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.087534    2567 scope.go:117] "RemoveContainer" containerID="b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"
Jul 06 01:48:06.087808 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:48:06.087792    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac\": container with ID starting with b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac not found: ID does not exist" containerID="b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"
Jul 06 01:48:06.087849 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.087818    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac"} err="failed to get container status \"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac\": rpc error: code = NotFound desc = could not find container \"b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac\": container with ID starting with b23c29defd785a1afa20c8c0869de38eedf3317f08f3947e49ddcbc2dd97dfac not found: ID does not exist"
Jul 06 01:48:06.098618 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.098593    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:48:06.105956 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.105929    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-17e353c4-7b8dbdb67d-v9t58"]
Jul 06 01:48:06.656840 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:06.656808    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" path="/var/lib/kubelet/pods/50d8967d-0e2e-40d8-a06f-5eeb52af361a/volumes"
Jul 06 01:48:07.757962 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:07.757931    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:48:33.571962 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.571886    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:48:33.572470 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.572450    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" containerName="maas-api"
Jul 06 01:48:33.572522 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.572475    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" containerName="maas-api"
Jul 06 01:48:33.572608 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.572596    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d8967d-0e2e-40d8-a06f-5eeb52af361a" containerName="maas-api"
Jul 06 01:48:33.575916 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.575898    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.578021 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.577999    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-3c8390c1-serving-cert\""
Jul 06 01:48:33.578154 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.578060    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:48:33.583167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.582939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:48:33.632101 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.632068    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.632266 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.632112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8x4j\" (UniqueName: \"kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.632266 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.632217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.733173 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.733143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.733321 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.733235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.733321 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.733260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8x4j\" (UniqueName: \"kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.733916 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.733891    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.735738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.735715    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.743194 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.743173    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8x4j\" (UniqueName: \"kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j\") pod \"maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") " pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:33.887964 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:33.887936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:34.044720 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:34.044692    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:48:34.046492 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:48:34.046466    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48635c2c_7ac8_4e3b_a76d_94f21f9f0074.slice/crio-1d052a4aaeb213193273198bd88d34e39f5bd8a5a754e823e4a7ac3f37f51b2b WatchSource:0}: Error finding container 1d052a4aaeb213193273198bd88d34e39f5bd8a5a754e823e4a7ac3f37f51b2b: Status 404 returned error can't find the container with id 1d052a4aaeb213193273198bd88d34e39f5bd8a5a754e823e4a7ac3f37f51b2b
Jul 06 01:48:34.213030 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:34.212951    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" event={"ID":"48635c2c-7ac8-4e3b-a76d-94f21f9f0074","Type":"ContainerStarted","Data":"1d052a4aaeb213193273198bd88d34e39f5bd8a5a754e823e4a7ac3f37f51b2b"}
Jul 06 01:48:35.219106 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:35.219073    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" event={"ID":"48635c2c-7ac8-4e3b-a76d-94f21f9f0074","Type":"ContainerStarted","Data":"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"}
Jul 06 01:48:35.219497 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:35.219199    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:35.235669 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:35.235627    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" podStartSLOduration=1.912300338 podStartE2EDuration="2.23561386s" totalImagesPullingTime="323.313522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:48:35.234421492 +0000 UTC m=+1715.079570903" watchObservedRunningTime="2026-07-06 01:48:35.23561386 +0000 UTC m=+1715.080763271"
Jul 06 01:48:38.662468 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:38.662389    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:48:41.229728 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:41.229700    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:48:51.764820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.764781    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:48:51.768907 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.768885    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.771982 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.771956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3c8390c1\"/\"kube-root-ca.crt\""
Jul 06 01:48:51.772110 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.771956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3c8390c1\"/\"openshift-service-ca.crt\""
Jul 06 01:48:51.772110 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.772080    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3c8390c1\"/\"e2e-lifecycle-model-3c8390c1-kserve-self-signed-certs\""
Jul 06 01:48:51.772314 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.772184    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3c8390c1\"/\"default-dockercfg-9cg85\""
Jul 06 01:48:51.776518 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.776493    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:48:51.888949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.888913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.888949 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.888950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.889231 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.888972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.889231 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.889063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.889231 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.889103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.889357 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.889231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wcls\" (UniqueName: \"kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.989968 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.989930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990172 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.989979    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990172 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wcls\" (UniqueName: \"kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990172 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990172 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990042    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990172 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990464 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990440    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990521 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990468    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.990521 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.990493    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.992279 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.992257    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.992633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.992615    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:51.999075 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:51.999050    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wcls\" (UniqueName: \"kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls\") pod \"e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") " pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:52.086742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:52.086660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:48:52.217498 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:52.217468    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:48:52.220196 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:48:52.220171    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90e2ebd5_c54b_4159_8b27_d7814b95f38c.slice/crio-910409ba2f228d885e0d3197ea5641626b1f299decb7cf18616e6e23fbf1637b WatchSource:0}: Error finding container 910409ba2f228d885e0d3197ea5641626b1f299decb7cf18616e6e23fbf1637b: Status 404 returned error can't find the container with id 910409ba2f228d885e0d3197ea5641626b1f299decb7cf18616e6e23fbf1637b
Jul 06 01:48:52.294777 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:52.294742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" event={"ID":"90e2ebd5-c54b-4159-8b27-d7814b95f38c","Type":"ContainerStarted","Data":"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"}
Jul 06 01:48:52.294915 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:52.294792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" event={"ID":"90e2ebd5-c54b-4159-8b27-d7814b95f38c","Type":"ContainerStarted","Data":"910409ba2f228d885e0d3197ea5641626b1f299decb7cf18616e6e23fbf1637b"}
Jul 06 01:48:52.311050 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:52.310988    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" podStartSLOduration=1.310966194 podStartE2EDuration="1.310966194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:48:52.309937623 +0000 UTC m=+1732.155087035" watchObservedRunningTime="2026-07-06 01:48:52.310966194 +0000 UTC m=+1732.156115606"
Jul 06 01:48:54.754700 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:48:54.754670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:49:02.087645 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:02.087605    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:02.088158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:02.087667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:02.100029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:02.100006    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:02.353060 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:02.352983    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:09.751217 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:09.751187    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:49:16.137533 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.137489    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:49:16.138055 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.137713    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerName="maas-api" containerID="cri-o://794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8" gracePeriod=30
Jul 06 01:49:16.224872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.224831    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.44:8443/health\": dial tcp 10.132.0.44:8443: connect: connection refused"
Jul 06 01:49:16.380817 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.380793    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:49:16.405014 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.404939    2567 generic.go:356] "Generic (PLEG): container finished" podID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerID="794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8" exitCode=0
Jul 06 01:49:16.405014 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.404986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" event={"ID":"48635c2c-7ac8-4e3b-a76d-94f21f9f0074","Type":"ContainerDied","Data":"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"}
Jul 06 01:49:16.405014 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.404992    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"
Jul 06 01:49:16.405239 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.405036    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk" event={"ID":"48635c2c-7ac8-4e3b-a76d-94f21f9f0074","Type":"ContainerDied","Data":"1d052a4aaeb213193273198bd88d34e39f5bd8a5a754e823e4a7ac3f37f51b2b"}
Jul 06 01:49:16.405239 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.405068    2567 scope.go:117] "RemoveContainer" containerID="794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"
Jul 06 01:49:16.413678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.413661    2567 scope.go:117] "RemoveContainer" containerID="794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"
Jul 06 01:49:16.413947 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:49:16.413921    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8\": container with ID starting with 794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8 not found: ID does not exist" containerID="794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"
Jul 06 01:49:16.414069 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.413952    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8"} err="failed to get container status \"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8\": rpc error: code = NotFound desc = could not find container \"794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8\": container with ID starting with 794c951081dec01b2d529f519dd62d1992e67e13c70edefb37a0b40a797b70b8 not found: ID does not exist"
Jul 06 01:49:16.500229 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.500196    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca\") pod \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") "
Jul 06 01:49:16.500396 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.500290    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls\") pod \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") "
Jul 06 01:49:16.500396 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.500339    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8x4j\" (UniqueName: \"kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j\") pod \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\" (UID: \"48635c2c-7ac8-4e3b-a76d-94f21f9f0074\") "
Jul 06 01:49:16.500615 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.500594    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48635c2c-7ac8-4e3b-a76d-94f21f9f0074" (UID: "48635c2c-7ac8-4e3b-a76d-94f21f9f0074"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:49:16.502492 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.502466    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48635c2c-7ac8-4e3b-a76d-94f21f9f0074" (UID: "48635c2c-7ac8-4e3b-a76d-94f21f9f0074"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:49:16.502676 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.502661    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j" (OuterVolumeSpecName: "kube-api-access-d8x4j") pod "48635c2c-7ac8-4e3b-a76d-94f21f9f0074" (UID: "48635c2c-7ac8-4e3b-a76d-94f21f9f0074"). InnerVolumeSpecName "kube-api-access-d8x4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:49:16.601872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.601843    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:16.601872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.601867    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:16.601872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.601877    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8x4j\" (UniqueName: \"kubernetes.io/projected/48635c2c-7ac8-4e3b-a76d-94f21f9f0074-kube-api-access-d8x4j\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:16.719709 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.719678    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:49:16.723579 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:16.723556    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-3c8390c1-5fc756f94f-sddjk"]
Jul 06 01:49:18.655970 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:18.655932    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" path="/var/lib/kubelet/pods/48635c2c-7ac8-4e3b-a76d-94f21f9f0074/volumes"
Jul 06 01:49:18.957762 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:18.957682    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:49:26.892998 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:26.892965    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:49:30.857620 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:30.857588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:49:41.913560 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:41.913511    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" podUID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" containerName="main" containerID="cri-o://3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069" gracePeriod=45
Jul 06 01:49:42.174242 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.174180    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:42.324775 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324733    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wcls\" (UniqueName: \"kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.324952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324786    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.324952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324808    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.324952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.324952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324878    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.324952 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.324911    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs\") pod \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\" (UID: \"90e2ebd5-c54b-4159-8b27-d7814b95f38c\") "
Jul 06 01:49:42.325253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.325183    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache" (OuterVolumeSpecName: "model-cache") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:49:42.325253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.325199    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:49:42.325253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.325208    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home" (OuterVolumeSpecName: "home") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:49:42.327235 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.327212    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm" (OuterVolumeSpecName: "dshm") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:49:42.327235 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.327231    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls" (OuterVolumeSpecName: "kube-api-access-7wcls") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "kube-api-access-7wcls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:49:42.327376 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.327265    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90e2ebd5-c54b-4159-8b27-d7814b95f38c" (UID: "90e2ebd5-c54b-4159-8b27-d7814b95f38c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426505    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wcls\" (UniqueName: \"kubernetes.io/projected/90e2ebd5-c54b-4159-8b27-d7814b95f38c-kube-api-access-7wcls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426538    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426549    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426558    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426565    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90e2ebd5-c54b-4159-8b27-d7814b95f38c-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.426593 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.426572    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90e2ebd5-c54b-4159-8b27-d7814b95f38c-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:49:42.533442 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.533406    2567 generic.go:356] "Generic (PLEG): container finished" podID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" containerID="3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069" exitCode=0
Jul 06 01:49:42.533605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.533450    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"
Jul 06 01:49:42.533605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.533494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" event={"ID":"90e2ebd5-c54b-4159-8b27-d7814b95f38c","Type":"ContainerDied","Data":"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"}
Jul 06 01:49:42.533605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.533537    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n" event={"ID":"90e2ebd5-c54b-4159-8b27-d7814b95f38c","Type":"ContainerDied","Data":"910409ba2f228d885e0d3197ea5641626b1f299decb7cf18616e6e23fbf1637b"}
Jul 06 01:49:42.533605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.533557    2567 scope.go:117] "RemoveContainer" containerID="3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"
Jul 06 01:49:42.542631 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.542611    2567 scope.go:117] "RemoveContainer" containerID="3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"
Jul 06 01:49:42.542916 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:49:42.542894    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069\": container with ID starting with 3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069 not found: ID does not exist" containerID="3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"
Jul 06 01:49:42.542978 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.542930    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069"} err="failed to get container status \"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069\": rpc error: code = NotFound desc = could not find container \"3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069\": container with ID starting with 3fe128aff6b0e1c4cfa077af6ca196eb1fe7d9a11e88df60285c8db5bb385069 not found: ID does not exist"
Jul 06 01:49:42.552907 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.552880    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:49:42.556860 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.556838    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3c8390c1/e2e-lifecycle-model-3c8390c1-kserve-755bb5db6c-n4v4n"]
Jul 06 01:49:42.655788 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:49:42.655754    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" path="/var/lib/kubelet/pods/90e2ebd5-c54b-4159-8b27-d7814b95f38c/volumes"
Jul 06 01:50:00.699820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:50:00.699791    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:50:00.713371 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:50:00.713351    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:50:06.351750 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:50:06.351701    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:50:14.860681 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:50:14.860644    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:50:26.758711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:50:26.758673    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:51:07.425834 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.425799    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426342    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" containerName="main"
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426359    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" containerName="main"
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426412    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerName="maas-api"
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426421    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerName="maas-api"
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426500    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e2ebd5-c54b-4159-8b27-d7814b95f38c" containerName="main"
Jul 06 01:51:07.428310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.426517    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="48635c2c-7ac8-4e3b-a76d-94f21f9f0074" containerName="maas-api"
Jul 06 01:51:07.429617 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.429600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.432162 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.432140    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-03c9df-serving-cert\""
Jul 06 01:51:07.432658 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.432642    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:51:07.440812 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.440791    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:51:07.515792 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.515761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8wxm\" (UniqueName: \"kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.515792 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.515797    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.515994 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.515908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.616476 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.616447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wxm\" (UniqueName: \"kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.616630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.616481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.616630 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.616527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.616630 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:51:07.616615    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-03c9df-serving-cert: secret "maas-api-e2e-inf-a-03c9df-serving-cert" not found
Jul 06 01:51:07.616734 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:51:07.616667    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls podName:f6853523-ab3d-4024-9232-1050a83be6ec nodeName:}" failed. No retries permitted until 2026-07-06 01:51:08.116652625 +0000 UTC m=+1867.961802015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls") pod "maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" (UID: "f6853523-ab3d-4024-9232-1050a83be6ec") : secret "maas-api-e2e-inf-a-03c9df-serving-cert" not found
Jul 06 01:51:07.617213 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.617193    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:07.625635 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:07.625610    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wxm\" (UniqueName: \"kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:08.121934 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.121900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:08.124467 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.124440    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") " pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:08.344299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.344264    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:08.483386 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.483351    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:51:08.485589 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:51:08.485567    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6853523_ab3d_4024_9232_1050a83be6ec.slice/crio-703dadf020c4d0cf4a21dfc59fd08753b14a77f643abfb4df8d6cfcc39e32ff7 WatchSource:0}: Error finding container 703dadf020c4d0cf4a21dfc59fd08753b14a77f643abfb4df8d6cfcc39e32ff7: Status 404 returned error can't find the container with id 703dadf020c4d0cf4a21dfc59fd08753b14a77f643abfb4df8d6cfcc39e32ff7
Jul 06 01:51:08.486824 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.486808    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 01:51:08.922485 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:08.922445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" event={"ID":"f6853523-ab3d-4024-9232-1050a83be6ec","Type":"ContainerStarted","Data":"703dadf020c4d0cf4a21dfc59fd08753b14a77f643abfb4df8d6cfcc39e32ff7"}
Jul 06 01:51:09.928286 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:09.928250    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" event={"ID":"f6853523-ab3d-4024-9232-1050a83be6ec","Type":"ContainerStarted","Data":"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"}
Jul 06 01:51:09.928664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:09.928313    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:09.944772 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:09.944726    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" podStartSLOduration=2.505277411 podStartE2EDuration="2.944710145s" totalImagesPullingTime="439.432734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:51:09.94308 +0000 UTC m=+1869.788229431" watchObservedRunningTime="2026-07-06 01:51:09.944710145 +0000 UTC m=+1869.789859617"
Jul 06 01:51:13.156786 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:13.156753    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:51:15.651835 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.651804    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:51:15.655576 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.655553    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.657783 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.657762    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-7c01b8-serving-cert\""
Jul 06 01:51:15.664204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.663793    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:51:15.791384 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.791355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft5xk\" (UniqueName: \"kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.791554 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.791404    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.791554 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.791486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.892620 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.892594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5xk\" (UniqueName: \"kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.892775 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.892645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.892775 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.892682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.892884 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:51:15.892795    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-7c01b8-serving-cert: secret "maas-api-e2e-inf-b-7c01b8-serving-cert" not found
Jul 06 01:51:15.892884 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:51:15.892848    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls podName:01d9242a-0425-4153-b70c-65e46ce85fa2 nodeName:}" failed. No retries permitted until 2026-07-06 01:51:16.392834293 +0000 UTC m=+1876.237983684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls") pod "maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" (UID: "01d9242a-0425-4153-b70c-65e46ce85fa2") : secret "maas-api-e2e-inf-b-7c01b8-serving-cert" not found
Jul 06 01:51:15.893309 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.893291    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.900916 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.900894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5xk\" (UniqueName: \"kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:15.939135 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:15.939054    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:51:16.397267 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:16.397234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:16.399934 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:16.399901    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c01b8-589b659765-k85b9\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") " pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:16.568297 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:16.568243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:16.700284 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:16.700252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:51:16.703319 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:51:16.703289    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d9242a_0425_4153_b70c_65e46ce85fa2.slice/crio-e91e630083c4c9fb805a8a598227100ecacac1f830d36244a6986be4c1c8e38d WatchSource:0}: Error finding container e91e630083c4c9fb805a8a598227100ecacac1f830d36244a6986be4c1c8e38d: Status 404 returned error can't find the container with id e91e630083c4c9fb805a8a598227100ecacac1f830d36244a6986be4c1c8e38d
Jul 06 01:51:16.960855 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:16.960778    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" event={"ID":"01d9242a-0425-4153-b70c-65e46ce85fa2","Type":"ContainerStarted","Data":"e91e630083c4c9fb805a8a598227100ecacac1f830d36244a6986be4c1c8e38d"}
Jul 06 01:51:17.966590 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:17.966553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" event={"ID":"01d9242a-0425-4153-b70c-65e46ce85fa2","Type":"ContainerStarted","Data":"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"}
Jul 06 01:51:17.967085 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:17.966613    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:17.983308 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:17.983265    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" podStartSLOduration=2.689624512 podStartE2EDuration="2.983250743s" totalImagesPullingTime="293.626231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:51:17.981818672 +0000 UTC m=+1877.826968095" watchObservedRunningTime="2026-07-06 01:51:17.983250743 +0000 UTC m=+1877.828400154"
Jul 06 01:51:23.976605 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:23.976525    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:51:24.553022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:24.552990    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:51:29.976262 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.976225    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:51:29.979857 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.979836    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:29.982436 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.982417    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-03c9df\"/\"kube-root-ca.crt\""
Jul 06 01:51:29.983320 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.983297    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-03c9df\"/\"openshift-service-ca.crt\""
Jul 06 01:51:29.983471 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.983320    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-03c9df\"/\"default-dockercfg-w5zdn\""
Jul 06 01:51:29.983471 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.983320    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-03c9df\"/\"test-model-03c9df-kserve-self-signed-certs\""
Jul 06 01:51:29.988228 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:29.988090    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:51:30.115827 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.115827 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.116059 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dds2r\" (UniqueName: \"kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.116059 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.116059 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.116059 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.115981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216744 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216751    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dds2r\" (UniqueName: \"kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.216914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.216895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.217235 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.217214    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.217298 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.217246    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.217298 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.217293    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.219289 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.219267    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.219572 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.219551    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.224945 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.224923    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dds2r\" (UniqueName: \"kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r\") pod \"test-model-03c9df-kserve-6c57b464cf-dqklc\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") " pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.291977 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.291915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:30.425338 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:30.425307    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:51:30.429564 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:51:30.429538    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d307527_66ea_4aba_a904_571404347f8d.slice/crio-e17aacefa57efaf0c5fe46f9fe36a499e5f258681c44572d768855d7f88acbcc WatchSource:0}: Error finding container e17aacefa57efaf0c5fe46f9fe36a499e5f258681c44572d768855d7f88acbcc: Status 404 returned error can't find the container with id e17aacefa57efaf0c5fe46f9fe36a499e5f258681c44572d768855d7f88acbcc
Jul 06 01:51:31.024600 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:31.024568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" event={"ID":"0d307527-66ea-4aba-a904-571404347f8d","Type":"ContainerStarted","Data":"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"}
Jul 06 01:51:31.024600 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:31.024605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" event={"ID":"0d307527-66ea-4aba-a904-571404347f8d","Type":"ContainerStarted","Data":"e17aacefa57efaf0c5fe46f9fe36a499e5f258681c44572d768855d7f88acbcc"}
Jul 06 01:51:31.041947 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:31.041897    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" podStartSLOduration=2.041883119 podStartE2EDuration="2.041883119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:51:31.040931846 +0000 UTC m=+1890.886081261" watchObservedRunningTime="2026-07-06 01:51:31.041883119 +0000 UTC m=+1890.887032531"
Jul 06 01:51:34.752793 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:34.752752    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:51:40.292575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:40.292521    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:40.292575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:40.292585    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:40.305011 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:40.304985    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:41.084608 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:41.084581    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:51:54.949543 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:51:54.949498    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:52:00.787416 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:00.787387    2567 scope.go:117] "RemoveContainer" containerID="ef452d06859f4e295c33a39dae2db98cd51c9d418b5d539b04cb34f073f72b66"
Jul 06 01:52:08.178684 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:08.178650    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:52:09.945777 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:09.945743    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:52:09.946204 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:09.945964    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" podUID="f6853523-ab3d-4024-9232-1050a83be6ec" containerName="maas-api" containerID="cri-o://3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb" gracePeriod=30
Jul 06 01:52:10.399664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.399639    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:52:10.544860 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.544767    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8wxm\" (UniqueName: \"kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm\") pod \"f6853523-ab3d-4024-9232-1050a83be6ec\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") "
Jul 06 01:52:10.544860 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.544846    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca\") pod \"f6853523-ab3d-4024-9232-1050a83be6ec\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") "
Jul 06 01:52:10.545104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.544907    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") pod \"f6853523-ab3d-4024-9232-1050a83be6ec\" (UID: \"f6853523-ab3d-4024-9232-1050a83be6ec\") "
Jul 06 01:52:10.545291 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.545267    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6853523-ab3d-4024-9232-1050a83be6ec" (UID: "f6853523-ab3d-4024-9232-1050a83be6ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:52:10.547095 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.547072    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm" (OuterVolumeSpecName: "kube-api-access-g8wxm") pod "f6853523-ab3d-4024-9232-1050a83be6ec" (UID: "f6853523-ab3d-4024-9232-1050a83be6ec"). InnerVolumeSpecName "kube-api-access-g8wxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:52:10.547095 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.547082    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6853523-ab3d-4024-9232-1050a83be6ec" (UID: "f6853523-ab3d-4024-9232-1050a83be6ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:52:10.646391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.646363    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6853523-ab3d-4024-9232-1050a83be6ec-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:10.646391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.646388    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8wxm\" (UniqueName: \"kubernetes.io/projected/f6853523-ab3d-4024-9232-1050a83be6ec-kube-api-access-g8wxm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:10.646391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:10.646398    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6853523-ab3d-4024-9232-1050a83be6ec-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:11.208057 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.208018    2567 generic.go:356] "Generic (PLEG): container finished" podID="f6853523-ab3d-4024-9232-1050a83be6ec" containerID="3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb" exitCode=0
Jul 06 01:52:11.208500 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.208080    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"
Jul 06 01:52:11.208500 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.208104    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" event={"ID":"f6853523-ab3d-4024-9232-1050a83be6ec","Type":"ContainerDied","Data":"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"}
Jul 06 01:52:11.208500 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.208187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq" event={"ID":"f6853523-ab3d-4024-9232-1050a83be6ec","Type":"ContainerDied","Data":"703dadf020c4d0cf4a21dfc59fd08753b14a77f643abfb4df8d6cfcc39e32ff7"}
Jul 06 01:52:11.208500 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.208212    2567 scope.go:117] "RemoveContainer" containerID="3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"
Jul 06 01:52:11.217532 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.217514    2567 scope.go:117] "RemoveContainer" containerID="3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"
Jul 06 01:52:11.217789 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:52:11.217771    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb\": container with ID starting with 3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb not found: ID does not exist" containerID="3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"
Jul 06 01:52:11.217841 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.217795    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb"} err="failed to get container status \"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb\": rpc error: code = NotFound desc = could not find container \"3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb\": container with ID starting with 3d293371371b3b96e16d2f459e2f7da4e0fd32bf5d5b9f26473556f9aabac6cb not found: ID does not exist"
Jul 06 01:52:11.224383 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.224359    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:52:11.227634 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:11.227615    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-03c9df-5b6f48c7dd-5zhsq"]
Jul 06 01:52:12.357596 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:12.357557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:52:12.656175 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:12.656135    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6853523-ab3d-4024-9232-1050a83be6ec" path="/var/lib/kubelet/pods/f6853523-ab3d-4024-9232-1050a83be6ec/volumes"
Jul 06 01:52:23.200067 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.200026    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" podUID="0d307527-66ea-4aba-a904-571404347f8d" containerName="main" containerID="cri-o://352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66" gracePeriod=45
Jul 06 01:52:23.462287 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.462224    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:52:23.554374 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554341    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554386    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554424    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dds2r\" (UniqueName: \"kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554471    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554512    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554535    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs\") pod \"0d307527-66ea-4aba-a904-571404347f8d\" (UID: \"0d307527-66ea-4aba-a904-571404347f8d\") "
Jul 06 01:52:23.554816 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554699    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:52:23.554903 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554801    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home" (OuterVolumeSpecName: "home") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:52:23.554903 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.554837    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:52:23.556818 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.556785    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm" (OuterVolumeSpecName: "dshm") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:52:23.556932 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.556905    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:52:23.556932 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.556907    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r" (OuterVolumeSpecName: "kube-api-access-dds2r") pod "0d307527-66ea-4aba-a904-571404347f8d" (UID: "0d307527-66ea-4aba-a904-571404347f8d"). InnerVolumeSpecName "kube-api-access-dds2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:52:23.656200 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656166    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:23.656200 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656190    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:23.656200 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656199    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d307527-66ea-4aba-a904-571404347f8d-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:23.656200 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656207    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:23.656450 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656234    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d307527-66ea-4aba-a904-571404347f8d-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:23.656450 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:23.656243    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dds2r\" (UniqueName: \"kubernetes.io/projected/0d307527-66ea-4aba-a904-571404347f8d-kube-api-access-dds2r\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:24.267575 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.267536    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d307527-66ea-4aba-a904-571404347f8d" containerID="352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66" exitCode=0
Jul 06 01:52:24.267986 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.267586    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"
Jul 06 01:52:24.267986 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.267624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" event={"ID":"0d307527-66ea-4aba-a904-571404347f8d","Type":"ContainerDied","Data":"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"}
Jul 06 01:52:24.267986 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.267670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc" event={"ID":"0d307527-66ea-4aba-a904-571404347f8d","Type":"ContainerDied","Data":"e17aacefa57efaf0c5fe46f9fe36a499e5f258681c44572d768855d7f88acbcc"}
Jul 06 01:52:24.267986 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.267686    2567 scope.go:117] "RemoveContainer" containerID="352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"
Jul 06 01:52:24.276550 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.276532    2567 scope.go:117] "RemoveContainer" containerID="352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"
Jul 06 01:52:24.276825 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:52:24.276804    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66\": container with ID starting with 352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66 not found: ID does not exist" containerID="352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"
Jul 06 01:52:24.276917 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.276832    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66"} err="failed to get container status \"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66\": rpc error: code = NotFound desc = could not find container \"352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66\": container with ID starting with 352cdc3ffdad34a539a84c521aae66a5e3f1767fb73c9b5764044787ac9c5b66 not found: ID does not exist"
Jul 06 01:52:24.287966 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.287940    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:52:24.289461 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.289442    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-03c9df/test-model-03c9df-kserve-6c57b464cf-dqklc"]
Jul 06 01:52:24.656531 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:24.656498    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d307527-66ea-4aba-a904-571404347f8d" path="/var/lib/kubelet/pods/0d307527-66ea-4aba-a904-571404347f8d/volumes"
Jul 06 01:52:52.732418 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:52.732326    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:52:52.732915 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:52.732576    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" podUID="01d9242a-0425-4153-b70c-65e46ce85fa2" containerName="maas-api" containerID="cri-o://2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704" gracePeriod=30
Jul 06 01:52:52.986428 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:52.986365    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:52:53.101804 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.101771    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca\") pod \"01d9242a-0425-4153-b70c-65e46ce85fa2\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") "
Jul 06 01:52:53.101985 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.101823    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft5xk\" (UniqueName: \"kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk\") pod \"01d9242a-0425-4153-b70c-65e46ce85fa2\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") "
Jul 06 01:52:53.101985 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.101857    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") pod \"01d9242a-0425-4153-b70c-65e46ce85fa2\" (UID: \"01d9242a-0425-4153-b70c-65e46ce85fa2\") "
Jul 06 01:52:53.102207 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.102179    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01d9242a-0425-4153-b70c-65e46ce85fa2" (UID: "01d9242a-0425-4153-b70c-65e46ce85fa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:52:53.104082 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.104059    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01d9242a-0425-4153-b70c-65e46ce85fa2" (UID: "01d9242a-0425-4153-b70c-65e46ce85fa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:52:53.104177 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.104075    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk" (OuterVolumeSpecName: "kube-api-access-ft5xk") pod "01d9242a-0425-4153-b70c-65e46ce85fa2" (UID: "01d9242a-0425-4153-b70c-65e46ce85fa2"). InnerVolumeSpecName "kube-api-access-ft5xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:52:53.203419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.203382    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d9242a-0425-4153-b70c-65e46ce85fa2-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:53.203419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.203413    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft5xk\" (UniqueName: \"kubernetes.io/projected/01d9242a-0425-4153-b70c-65e46ce85fa2-kube-api-access-ft5xk\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:53.203419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.203426    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d9242a-0425-4153-b70c-65e46ce85fa2-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:52:53.397558 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.397522    2567 generic.go:356] "Generic (PLEG): container finished" podID="01d9242a-0425-4153-b70c-65e46ce85fa2" containerID="2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704" exitCode=0
Jul 06 01:52:53.397731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.397566    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" event={"ID":"01d9242a-0425-4153-b70c-65e46ce85fa2","Type":"ContainerDied","Data":"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"}
Jul 06 01:52:53.397731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.397570    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"
Jul 06 01:52:53.397731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.397607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9" event={"ID":"01d9242a-0425-4153-b70c-65e46ce85fa2","Type":"ContainerDied","Data":"e91e630083c4c9fb805a8a598227100ecacac1f830d36244a6986be4c1c8e38d"}
Jul 06 01:52:53.397731 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.397629    2567 scope.go:117] "RemoveContainer" containerID="2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"
Jul 06 01:52:53.407038 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.407014    2567 scope.go:117] "RemoveContainer" containerID="2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"
Jul 06 01:52:53.407381 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:52:53.407351    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704\": container with ID starting with 2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704 not found: ID does not exist" containerID="2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"
Jul 06 01:52:53.407490 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.407393    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704"} err="failed to get container status \"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704\": rpc error: code = NotFound desc = could not find container \"2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704\": container with ID starting with 2f2416ca3a842aeae9c417b814cf58238f88c785d8204b7355b9b601869b7704 not found: ID does not exist"
Jul 06 01:52:53.419918 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.419866    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:52:53.421470 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:53.421448    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c01b8-589b659765-k85b9"]
Jul 06 01:52:54.656309 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:54.656275    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01d9242a-0425-4153-b70c-65e46ce85fa2" path="/var/lib/kubelet/pods/01d9242a-0425-4153-b70c-65e46ce85fa2/volumes"
Jul 06 01:52:55.855378 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:52:55.855345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:02.054211 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054177    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:02.054684 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054670    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6853523-ab3d-4024-9232-1050a83be6ec" containerName="maas-api"
Jul 06 01:53:02.054727 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054689    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6853523-ab3d-4024-9232-1050a83be6ec" containerName="maas-api"
Jul 06 01:53:02.054727 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054703    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d307527-66ea-4aba-a904-571404347f8d" containerName="main"
Jul 06 01:53:02.054727 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054712    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d307527-66ea-4aba-a904-571404347f8d" containerName="main"
Jul 06 01:53:02.054829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054734    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01d9242a-0425-4153-b70c-65e46ce85fa2" containerName="maas-api"
Jul 06 01:53:02.054829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054744    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01d9242a-0425-4153-b70c-65e46ce85fa2" containerName="maas-api"
Jul 06 01:53:02.054888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054833    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d307527-66ea-4aba-a904-571404347f8d" containerName="main"
Jul 06 01:53:02.054888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054845    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01d9242a-0425-4153-b70c-65e46ce85fa2" containerName="maas-api"
Jul 06 01:53:02.054888 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.054854    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6853523-ab3d-4024-9232-1050a83be6ec" containerName="maas-api"
Jul 06 01:53:02.059527 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.059507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.062265 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.062233    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-eb606b-serving-cert\""
Jul 06 01:53:02.062265 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.062257    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:53:02.068214 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.068193    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:02.178082 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.178041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.178283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.178094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx24c\" (UniqueName: \"kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.178283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.178117    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.279059 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.279018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.279299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.279078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx24c\" (UniqueName: \"kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.279299 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.279109    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.279299 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:02.279204    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-eb606b-serving-cert: secret "maas-api-e2e-api-a-eb606b-serving-cert" not found
Jul 06 01:53:02.279299 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:02.279296    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls podName:f151d403-b884-40c6-b6d1-5662055a5702 nodeName:}" failed. No retries permitted until 2026-07-06 01:53:02.779279066 +0000 UTC m=+1982.624428456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls") pod "maas-api-e2e-api-a-eb606b-76984b8849-gctsm" (UID: "f151d403-b884-40c6-b6d1-5662055a5702") : secret "maas-api-e2e-api-a-eb606b-serving-cert" not found
Jul 06 01:53:02.279820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.279796    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.287998 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.287968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx24c\" (UniqueName: \"kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.783642 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.783600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.786178 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.786148    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") pod \"maas-api-e2e-api-a-eb606b-76984b8849-gctsm\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") " pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:02.971009 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:02.970972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:03.104761 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:03.104731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:03.107274 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:53:03.107243    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf151d403_b884_40c6_b6d1_5662055a5702.slice/crio-9167d61224c9e3d77a83a87d90fa63ab98934687d0fa1ed1aa22a3b2098d1482 WatchSource:0}: Error finding container 9167d61224c9e3d77a83a87d90fa63ab98934687d0fa1ed1aa22a3b2098d1482: Status 404 returned error can't find the container with id 9167d61224c9e3d77a83a87d90fa63ab98934687d0fa1ed1aa22a3b2098d1482
Jul 06 01:53:03.443984 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:03.443943    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" event={"ID":"f151d403-b884-40c6-b6d1-5662055a5702","Type":"ContainerStarted","Data":"9167d61224c9e3d77a83a87d90fa63ab98934687d0fa1ed1aa22a3b2098d1482"}
Jul 06 01:53:04.449486 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:04.449452    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" event={"ID":"f151d403-b884-40c6-b6d1-5662055a5702","Type":"ContainerStarted","Data":"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"}
Jul 06 01:53:04.449851 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:04.449576    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:04.465210 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:04.465161    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" podStartSLOduration=2.09383774 podStartE2EDuration="2.46514373s" totalImagesPullingTime="371.30599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:53:04.464147467 +0000 UTC m=+1984.309296912" watchObservedRunningTime="2026-07-06 01:53:04.46514373 +0000 UTC m=+1984.310293140"
Jul 06 01:53:09.651798 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:09.651766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:10.085945 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.085858    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:10.089640 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.089619    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.093300 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.093271    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-43d931-serving-cert\""
Jul 06 01:53:10.103022 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.102999    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:10.148696 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.148665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.148696 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.148703    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.148911 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.148781    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swm69\" (UniqueName: \"kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.249246 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.249215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.249419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.249249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.249419 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.249282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swm69\" (UniqueName: \"kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.249419 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:10.249388    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-43d931-serving-cert: secret "maas-api-e2e-api-b-43d931-serving-cert" not found
Jul 06 01:53:10.249594 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:10.249461    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls podName:fb1d4b91-4f29-4e5a-8267-182acbed5036 nodeName:}" failed. No retries permitted until 2026-07-06 01:53:10.749442139 +0000 UTC m=+1990.594591533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls") pod "maas-api-e2e-api-b-43d931-7d449674d5-km5tg" (UID: "fb1d4b91-4f29-4e5a-8267-182acbed5036") : secret "maas-api-e2e-api-b-43d931-serving-cert" not found
Jul 06 01:53:10.249845 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.249818    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.257558 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.257531    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swm69\" (UniqueName: \"kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.459500 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.459466    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:10.755429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.755338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:10.757723 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:10.757705    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") pod \"maas-api-e2e-api-b-43d931-7d449674d5-km5tg\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") " pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:11.003494 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:11.003459    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:11.139621 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:11.139592    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:11.143272 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:53:11.143239    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1d4b91_4f29_4e5a_8267_182acbed5036.slice/crio-6ee700b9fdc7a517c64df412c33206575cdd83ab1e4812f6d8dac8ad05308f49 WatchSource:0}: Error finding container 6ee700b9fdc7a517c64df412c33206575cdd83ab1e4812f6d8dac8ad05308f49: Status 404 returned error can't find the container with id 6ee700b9fdc7a517c64df412c33206575cdd83ab1e4812f6d8dac8ad05308f49
Jul 06 01:53:11.483979 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:11.483948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" event={"ID":"fb1d4b91-4f29-4e5a-8267-182acbed5036","Type":"ContainerStarted","Data":"6ee700b9fdc7a517c64df412c33206575cdd83ab1e4812f6d8dac8ad05308f49"}
Jul 06 01:53:12.489901 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:12.489865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" event={"ID":"fb1d4b91-4f29-4e5a-8267-182acbed5036","Type":"ContainerStarted","Data":"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"}
Jul 06 01:53:12.490333 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:12.489974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:12.504964 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:12.504920    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" podStartSLOduration=2.086232939 podStartE2EDuration="2.504905765s" totalImagesPullingTime="418.672826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:53:12.504247533 +0000 UTC m=+1992.349396976" watchObservedRunningTime="2026-07-06 01:53:12.504905765 +0000 UTC m=+1992.350055177"
Jul 06 01:53:18.500787 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:18.500754    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:20.656774 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:20.656746    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:23.246364 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.246322    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:23.247178 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.247143    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" podUID="f151d403-b884-40c6-b6d1-5662055a5702" containerName="maas-api" containerID="cri-o://b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21" gracePeriod=30
Jul 06 01:53:23.507933 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.507861    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:23.544158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.544094    2567 generic.go:356] "Generic (PLEG): container finished" podID="f151d403-b884-40c6-b6d1-5662055a5702" containerID="b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21" exitCode=0
Jul 06 01:53:23.544158 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.544157    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"
Jul 06 01:53:23.544352 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.544173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" event={"ID":"f151d403-b884-40c6-b6d1-5662055a5702","Type":"ContainerDied","Data":"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"}
Jul 06 01:53:23.544352 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.544230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm" event={"ID":"f151d403-b884-40c6-b6d1-5662055a5702","Type":"ContainerDied","Data":"9167d61224c9e3d77a83a87d90fa63ab98934687d0fa1ed1aa22a3b2098d1482"}
Jul 06 01:53:23.544352 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.544261    2567 scope.go:117] "RemoveContainer" containerID="b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"
Jul 06 01:53:23.552753 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.552733    2567 scope.go:117] "RemoveContainer" containerID="b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"
Jul 06 01:53:23.553030 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:23.553012    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21\": container with ID starting with b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21 not found: ID does not exist" containerID="b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"
Jul 06 01:53:23.553091 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.553039    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21"} err="failed to get container status \"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21\": rpc error: code = NotFound desc = could not find container \"b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21\": container with ID starting with b991f7ca700efd4c7dd0019cce330ef5a49052c41dd79b2d12da0f849b7cbf21 not found: ID does not exist"
Jul 06 01:53:23.667561 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.667534    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx24c\" (UniqueName: \"kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c\") pod \"f151d403-b884-40c6-b6d1-5662055a5702\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") "
Jul 06 01:53:23.667752 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.667599    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") pod \"f151d403-b884-40c6-b6d1-5662055a5702\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") "
Jul 06 01:53:23.667752 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.667628    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca\") pod \"f151d403-b884-40c6-b6d1-5662055a5702\" (UID: \"f151d403-b884-40c6-b6d1-5662055a5702\") "
Jul 06 01:53:23.668024 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.668002    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f151d403-b884-40c6-b6d1-5662055a5702" (UID: "f151d403-b884-40c6-b6d1-5662055a5702"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:53:23.669958 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.669924    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f151d403-b884-40c6-b6d1-5662055a5702" (UID: "f151d403-b884-40c6-b6d1-5662055a5702"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:53:23.670073 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.669948    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c" (OuterVolumeSpecName: "kube-api-access-nx24c") pod "f151d403-b884-40c6-b6d1-5662055a5702" (UID: "f151d403-b884-40c6-b6d1-5662055a5702"). InnerVolumeSpecName "kube-api-access-nx24c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:53:23.769167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.769068    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx24c\" (UniqueName: \"kubernetes.io/projected/f151d403-b884-40c6-b6d1-5662055a5702-kube-api-access-nx24c\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:23.769167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.769094    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f151d403-b884-40c6-b6d1-5662055a5702-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:23.769167 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.769103    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f151d403-b884-40c6-b6d1-5662055a5702-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:23.864735 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.864648    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:23.867154 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:23.867112    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-eb606b-76984b8849-gctsm"]
Jul 06 01:53:24.656169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:24.656107    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f151d403-b884-40c6-b6d1-5662055a5702" path="/var/lib/kubelet/pods/f151d403-b884-40c6-b6d1-5662055a5702/volumes"
Jul 06 01:53:30.272898 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.272862    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:30.273390 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.273163    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" podUID="fb1d4b91-4f29-4e5a-8267-182acbed5036" containerName="maas-api" containerID="cri-o://f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72" gracePeriod=30
Jul 06 01:53:30.522035 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.522011    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:30.578539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.578454    2567 generic.go:356] "Generic (PLEG): container finished" podID="fb1d4b91-4f29-4e5a-8267-182acbed5036" containerID="f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72" exitCode=0
Jul 06 01:53:30.578539 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.578500    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"
Jul 06 01:53:30.578734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.578539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" event={"ID":"fb1d4b91-4f29-4e5a-8267-182acbed5036","Type":"ContainerDied","Data":"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"}
Jul 06 01:53:30.578734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.578581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg" event={"ID":"fb1d4b91-4f29-4e5a-8267-182acbed5036","Type":"ContainerDied","Data":"6ee700b9fdc7a517c64df412c33206575cdd83ab1e4812f6d8dac8ad05308f49"}
Jul 06 01:53:30.578734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.578601    2567 scope.go:117] "RemoveContainer" containerID="f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"
Jul 06 01:53:30.587002 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.586982    2567 scope.go:117] "RemoveContainer" containerID="f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"
Jul 06 01:53:30.587294 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:30.587274    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72\": container with ID starting with f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72 not found: ID does not exist" containerID="f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"
Jul 06 01:53:30.587370 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.587303    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72"} err="failed to get container status \"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72\": rpc error: code = NotFound desc = could not find container \"f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72\": container with ID starting with f400403ee5533a19f7afe76ca9c16445dcf7b0727968a412212ad05909c88b72 not found: ID does not exist"
Jul 06 01:53:30.627815 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.627785    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") pod \"fb1d4b91-4f29-4e5a-8267-182acbed5036\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") "
Jul 06 01:53:30.627972 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.627856    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swm69\" (UniqueName: \"kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69\") pod \"fb1d4b91-4f29-4e5a-8267-182acbed5036\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") "
Jul 06 01:53:30.627972 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.627876    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca\") pod \"fb1d4b91-4f29-4e5a-8267-182acbed5036\" (UID: \"fb1d4b91-4f29-4e5a-8267-182acbed5036\") "
Jul 06 01:53:30.628277 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.628252    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb1d4b91-4f29-4e5a-8267-182acbed5036" (UID: "fb1d4b91-4f29-4e5a-8267-182acbed5036"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:53:30.630000 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.629977    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb1d4b91-4f29-4e5a-8267-182acbed5036" (UID: "fb1d4b91-4f29-4e5a-8267-182acbed5036"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:53:30.630079 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.630058    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69" (OuterVolumeSpecName: "kube-api-access-swm69") pod "fb1d4b91-4f29-4e5a-8267-182acbed5036" (UID: "fb1d4b91-4f29-4e5a-8267-182acbed5036"). InnerVolumeSpecName "kube-api-access-swm69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:53:30.729024 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.728998    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1d4b91-4f29-4e5a-8267-182acbed5036-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:30.729024 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.729023    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swm69\" (UniqueName: \"kubernetes.io/projected/fb1d4b91-4f29-4e5a-8267-182acbed5036-kube-api-access-swm69\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:30.729233 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.729033    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d4b91-4f29-4e5a-8267-182acbed5036-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:53:30.892911 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.892860    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:30.894711 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:30.894688    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-43d931-7d449674d5-km5tg"]
Jul 06 01:53:32.461354 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:32.461320    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:32.655795 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:32.655761    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1d4b91-4f29-4e5a-8267-182acbed5036" path="/var/lib/kubelet/pods/fb1d4b91-4f29-4e5a-8267-182acbed5036/volumes"
Jul 06 01:53:40.318959 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.318925    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319291    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d4b91-4f29-4e5a-8267-182acbed5036" containerName="maas-api"
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319306    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d4b91-4f29-4e5a-8267-182acbed5036" containerName="maas-api"
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319324    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f151d403-b884-40c6-b6d1-5662055a5702" containerName="maas-api"
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319329    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f151d403-b884-40c6-b6d1-5662055a5702" containerName="maas-api"
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319393    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d4b91-4f29-4e5a-8267-182acbed5036" containerName="maas-api"
Jul 06 01:53:40.319429 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.319402    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f151d403-b884-40c6-b6d1-5662055a5702" containerName="maas-api"
Jul 06 01:53:40.322534 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.322516    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.324867 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.324811    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-5afb9c-serving-cert\""
Jul 06 01:53:40.325221 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.325204    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 01:53:40.330919 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.330881    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:53:40.407183 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.407149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.407344 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.407190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbd2j\" (UniqueName: \"kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.407414 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.407338    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.508177 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.508142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.508322 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.508189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.508322 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:40.508276    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-5afb9c-serving-cert: secret "maas-api-e2e-shared-a-5afb9c-serving-cert" not found
Jul 06 01:53:40.508322 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:53:40.508322    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls podName:487d1953-dd8e-4674-9127-362b513fc62c nodeName:}" failed. No retries permitted until 2026-07-06 01:53:41.008306668 +0000 UTC m=+2020.853456057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls") pod "maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" (UID: "487d1953-dd8e-4674-9127-362b513fc62c") : secret "maas-api-e2e-shared-a-5afb9c-serving-cert" not found
Jul 06 01:53:40.508437 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.508350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbd2j\" (UniqueName: \"kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.508767 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.508749    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:40.520734 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:40.520705    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbd2j\" (UniqueName: \"kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:41.013517 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:41.013486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:41.016050 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:41.016019    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") pod \"maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") " pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:41.234812 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:41.234784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:41.367152 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:41.367110    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:53:41.370037 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:53:41.370013    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487d1953_dd8e_4674_9127_362b513fc62c.slice/crio-2d3fddfc78be708b6e130c156d72a552311886430e25659857190c48a426ed95 WatchSource:0}: Error finding container 2d3fddfc78be708b6e130c156d72a552311886430e25659857190c48a426ed95: Status 404 returned error can't find the container with id 2d3fddfc78be708b6e130c156d72a552311886430e25659857190c48a426ed95
Jul 06 01:53:41.631996 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:41.631954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" event={"ID":"487d1953-dd8e-4674-9127-362b513fc62c","Type":"ContainerStarted","Data":"2d3fddfc78be708b6e130c156d72a552311886430e25659857190c48a426ed95"}
Jul 06 01:53:42.638165 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:42.638102    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" event={"ID":"487d1953-dd8e-4674-9127-362b513fc62c","Type":"ContainerStarted","Data":"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"}
Jul 06 01:53:42.638798 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:42.638391    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:42.657954 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:42.657899    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" podStartSLOduration=2.338714344 podStartE2EDuration="2.657885489s" totalImagesPullingTime="319.171145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:53:42.655991106 +0000 UTC m=+2022.501140538" watchObservedRunningTime="2026-07-06 01:53:42.657885489 +0000 UTC m=+2022.503034942"
Jul 06 01:53:45.550893 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:45.550858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:48.384358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.384324    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:53:48.388604 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.388581    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.391107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.391076    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-ed9e8f-serving-cert\""
Jul 06 01:53:48.398793 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.398689    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:53:48.471306 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.471273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdff5\" (UniqueName: \"kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.471491 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.471322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.471491 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.471412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.572330 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.572298    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdff5\" (UniqueName: \"kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.572483 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.572339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.572483 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.572465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.573034 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.573010    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.575112 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.575088    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.580781 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.580757    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdff5\" (UniqueName: \"kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5\") pod \"maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") " pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.648787 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.648713    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:53:48.701417 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.701388    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:48.845407 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:48.845373    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:53:48.848506 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:53:48.848479    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a10e6f_fedf_4081_b1ac_68d90c964c20.slice/crio-6750f848e9b244299249aa689868cbb81a940c2fa217b5ff41842f28804e4d1e WatchSource:0}: Error finding container 6750f848e9b244299249aa689868cbb81a940c2fa217b5ff41842f28804e4d1e: Status 404 returned error can't find the container with id 6750f848e9b244299249aa689868cbb81a940c2fa217b5ff41842f28804e4d1e
Jul 06 01:53:49.675185 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:49.675148    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" event={"ID":"f3a10e6f-fedf-4081-b1ac-68d90c964c20","Type":"ContainerStarted","Data":"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"}
Jul 06 01:53:49.675613 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:49.675199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" event={"ID":"f3a10e6f-fedf-4081-b1ac-68d90c964c20","Type":"ContainerStarted","Data":"6750f848e9b244299249aa689868cbb81a940c2fa217b5ff41842f28804e4d1e"}
Jul 06 01:53:49.675613 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:49.675230    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:53:49.691998 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:49.691951    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" podStartSLOduration=1.344753238 podStartE2EDuration="1.69193702s" totalImagesPullingTime="347.183782ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:53:49.69022154 +0000 UTC m=+2029.535370952" watchObservedRunningTime="2026-07-06 01:53:49.69193702 +0000 UTC m=+2029.537086431"
Jul 06 01:53:54.154992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:54.154957    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:53:55.685033 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:53:55.684997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:54:02.476872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.476839    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:54:02.481633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.481614    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.484919 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.484898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"auth-test-model-5afb9c-kserve-self-signed-certs\""
Jul 06 01:54:02.485046 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.484942    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"kube-root-ca.crt\""
Jul 06 01:54:02.485140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.484915    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"default-dockercfg-fbqxk\""
Jul 06 01:54:02.485140 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.484898    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"openshift-service-ca.crt\""
Jul 06 01:54:02.488532 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.488509    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:54:02.592555 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.592742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp4kx\" (UniqueName: \"kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.592742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592588    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.592742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592614    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.592742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.592742 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.592720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693703 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693703 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp4kx\" (UniqueName: \"kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693953 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693953 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693953 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693762    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.693953 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.693796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.694151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.694045    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.694151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.694098    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.694151 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.694140    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.696358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.696325    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.696463 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.696443    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.701326 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.701308    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp4kx\" (UniqueName: \"kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx\") pod \"auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") " pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.793217 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.793157    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:02.925794 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:02.925771    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:54:02.927794 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:54:02.927768    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5409c72_fc51_4482_afff_26c6fa7ba2d6.slice/crio-815cb3074578f36e9de67514e2369eea175a4a5cd47d8625ac915d8e26d126eb WatchSource:0}: Error finding container 815cb3074578f36e9de67514e2369eea175a4a5cd47d8625ac915d8e26d126eb: Status 404 returned error can't find the container with id 815cb3074578f36e9de67514e2369eea175a4a5cd47d8625ac915d8e26d126eb
Jul 06 01:54:03.740139 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:03.740094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" event={"ID":"e5409c72-fc51-4482-afff-26c6fa7ba2d6","Type":"ContainerStarted","Data":"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"}
Jul 06 01:54:03.740499 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:03.740151    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" event={"ID":"e5409c72-fc51-4482-afff-26c6fa7ba2d6","Type":"ContainerStarted","Data":"815cb3074578f36e9de67514e2369eea175a4a5cd47d8625ac915d8e26d126eb"}
Jul 06 01:54:03.757913 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:03.757856    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" podStartSLOduration=1.757841416 podStartE2EDuration="1.757841416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:54:03.757228759 +0000 UTC m=+2043.602378180" watchObservedRunningTime="2026-07-06 01:54:03.757841416 +0000 UTC m=+2043.602990828"
Jul 06 01:54:06.055911 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:06.055877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:54:12.794004 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:12.793970    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:12.794484 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:12.794030    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:12.806606 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:12.806584    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:13.794582 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:13.794551    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:54:42.955790 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:42.955713    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:54:56.055371 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:54:56.055336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:55:00.726071 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:00.726040    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:55:00.739386 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:00.739365    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 01:55:11.656878 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:11.656845    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:55:26.749558 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:26.749523    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:55:43.553044 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:43.553012    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:55:59.357274 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:55:59.357192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:56:16.058530 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:16.058497    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:56:33.551544 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:33.551510    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:56:37.958960 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:37.958927    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:56:49.566107 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.566077    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:56:49.569438 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.569423    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.571646 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.571627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"sub-test-model-5afb9c-kserve-self-signed-certs\""
Jul 06 01:56:49.578476 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.578450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:56:49.698092 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.698283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tfmr\" (UniqueName: \"kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.698283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.698283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.698415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.698415 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.698336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799099 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tfmr\" (UniqueName: \"kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799303 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799600 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799552    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799622    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.799678 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.799651    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.801591 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.801571    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.801879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.801858    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.807383 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.807362    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tfmr\" (UniqueName: \"kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr\") pod \"sub-test-model-5afb9c-kserve-598877f595-qbk9f\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") " pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:49.881269 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:49.881247    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:50.020238 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:50.020206    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:56:50.023805 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:56:50.023764    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1fa2b17_3d35_4172_949d_4c9ca8debb4a.slice/crio-ddec9ec9a234613d7d355802b8f96e17a33a91cc54a5fd370f560c63e3da1636 WatchSource:0}: Error finding container ddec9ec9a234613d7d355802b8f96e17a33a91cc54a5fd370f560c63e3da1636: Status 404 returned error can't find the container with id ddec9ec9a234613d7d355802b8f96e17a33a91cc54a5fd370f560c63e3da1636
Jul 06 01:56:50.460735 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:50.458315    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:56:50.517201 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:50.517153    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" event={"ID":"d1fa2b17-3d35-4172-949d-4c9ca8debb4a","Type":"ContainerStarted","Data":"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"}
Jul 06 01:56:50.517201 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:50.517202    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" event={"ID":"d1fa2b17-3d35-4172-949d-4c9ca8debb4a","Type":"ContainerStarted","Data":"ddec9ec9a234613d7d355802b8f96e17a33a91cc54a5fd370f560c63e3da1636"}
Jul 06 01:56:50.532766 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:50.532715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" podStartSLOduration=1.532701426 podStartE2EDuration="1.532701426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:56:50.531777287 +0000 UTC m=+2210.376926700" watchObservedRunningTime="2026-07-06 01:56:50.532701426 +0000 UTC m=+2210.377850838"
Jul 06 01:56:52.979370 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:52.979309    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" podUID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" containerName="main" containerID="cri-o://1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03" gracePeriod=45
Jul 06 01:56:53.223686 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.223665    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:56:53.335437 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335348    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335437 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335394    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335437 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335417    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335677 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335518    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335694    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335738 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335698    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:56:53.335842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335745    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp4kx\" (UniqueName: \"kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx\") pod \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\" (UID: \"e5409c72-fc51-4482-afff-26c6fa7ba2d6\") "
Jul 06 01:56:53.335842 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335825    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home" (OuterVolumeSpecName: "home") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:56:53.335981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.335942    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:56:53.336244 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.336010    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.336244 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.336022    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.336244 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.336035    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.337871 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.337848    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm" (OuterVolumeSpecName: "dshm") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:56:53.337955 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.337848    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:56:53.337992 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.337959    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx" (OuterVolumeSpecName: "kube-api-access-kp4kx") pod "e5409c72-fc51-4482-afff-26c6fa7ba2d6" (UID: "e5409c72-fc51-4482-afff-26c6fa7ba2d6"). InnerVolumeSpecName "kube-api-access-kp4kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:56:53.436763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.436709    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp4kx\" (UniqueName: \"kubernetes.io/projected/e5409c72-fc51-4482-afff-26c6fa7ba2d6-kube-api-access-kp4kx\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.436763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.436754    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5409c72-fc51-4482-afff-26c6fa7ba2d6-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.436763 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.436763    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5409c72-fc51-4482-afff-26c6fa7ba2d6-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:56:53.531646 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.531608    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" containerID="1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03" exitCode=0
Jul 06 01:56:53.531810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.531655    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"
Jul 06 01:56:53.531810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.531677    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" event={"ID":"e5409c72-fc51-4482-afff-26c6fa7ba2d6","Type":"ContainerDied","Data":"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"}
Jul 06 01:56:53.531810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.531717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn" event={"ID":"e5409c72-fc51-4482-afff-26c6fa7ba2d6","Type":"ContainerDied","Data":"815cb3074578f36e9de67514e2369eea175a4a5cd47d8625ac915d8e26d126eb"}
Jul 06 01:56:53.531810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.531739    2567 scope.go:117] "RemoveContainer" containerID="1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"
Jul 06 01:56:53.541034 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.541018    2567 scope.go:117] "RemoveContainer" containerID="1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"
Jul 06 01:56:53.541294 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:56:53.541276    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03\": container with ID starting with 1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03 not found: ID does not exist" containerID="1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"
Jul 06 01:56:53.541358 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.541299    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03"} err="failed to get container status \"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03\": rpc error: code = NotFound desc = could not find container \"1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03\": container with ID starting with 1977f29fb4cae9ee0e3445ce85628dab6d5b51a052f42acdb2f45331ef8cdf03 not found: ID does not exist"
Jul 06 01:56:53.551795 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.551771    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:56:53.554987 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:53.554966    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/auth-test-model-5afb9c-kserve-6954b55b4c-q7fvn"]
Jul 06 01:56:54.656645 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:54.656613    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" path="/var/lib/kubelet/pods/e5409c72-fc51-4482-afff-26c6fa7ba2d6/volumes"
Jul 06 01:56:59.881448 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:59.881417    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:59.881879 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:59.881464    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:56:59.894002 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:56:59.893971    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:57:00.579073 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:00.579047    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:57:01.456709 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:01.456675    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:57:10.564047 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.564011    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:57:10.564509 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.564400    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" containerName="main"
Jul 06 01:57:10.564509 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.564412    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" containerName="main"
Jul 06 01:57:10.564509 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.564495    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5409c72-fc51-4482-afff-26c6fa7ba2d6" containerName="main"
Jul 06 01:57:10.567293 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.567274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.569809 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.569774    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"default-dockercfg-rxhjd\""
Jul 06 01:57:10.569932 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.569853    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"openshift-service-ca.crt\""
Jul 06 01:57:10.570872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.570798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"sub-test-model-ed9e8f-kserve-self-signed-certs\""
Jul 06 01:57:10.570872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.570856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"kube-root-ca.crt\""
Jul 06 01:57:10.577646 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.577621    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:57:10.682281 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.682281 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682283    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.682546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.682546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.682546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.682546 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.682521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd5wc\" (UniqueName: \"kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.783699 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.783896 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.783896 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.783896 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd5wc\" (UniqueName: \"kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.784066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.784066 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.783992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.784260 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.784229    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.784385 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.784371    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.784430 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.784388    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.786181 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.786157    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.786360 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.786344    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.791689 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.791662    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd5wc\" (UniqueName: \"kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc\") pod \"sub-test-model-ed9e8f-kserve-855d699765-whx8q\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:10.880904 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:10.880869    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:11.017011 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:11.016983    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:57:11.019061 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:57:11.019034    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ed585ea_bd1b_492b_89b8_416b1219b96d.slice/crio-31c0705b39ebdd2593e39af827bb9a14d6bddb459f07d524ce72b5354f691863 WatchSource:0}: Error finding container 31c0705b39ebdd2593e39af827bb9a14d6bddb459f07d524ce72b5354f691863: Status 404 returned error can't find the container with id 31c0705b39ebdd2593e39af827bb9a14d6bddb459f07d524ce72b5354f691863
Jul 06 01:57:11.620549 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:11.620498    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" event={"ID":"9ed585ea-bd1b-492b-89b8-416b1219b96d","Type":"ContainerStarted","Data":"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"}
Jul 06 01:57:11.620549 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:11.620554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" event={"ID":"9ed585ea-bd1b-492b-89b8-416b1219b96d","Type":"ContainerStarted","Data":"31c0705b39ebdd2593e39af827bb9a14d6bddb459f07d524ce72b5354f691863"}
Jul 06 01:57:11.638612 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:11.638563    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" podStartSLOduration=1.638548642 podStartE2EDuration="1.638548642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:57:11.63686206 +0000 UTC m=+2231.482011473" watchObservedRunningTime="2026-07-06 01:57:11.638548642 +0000 UTC m=+2231.483698053"
Jul 06 01:57:15.259233 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:15.259197    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:57:20.881347 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:20.881303    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:20.881843 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:20.881366    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:20.893617 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:20.893599    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:21.678560 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:21.678476    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:57:34.267664 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:34.267617    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:57:52.065063 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:52.065023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:57:58.057283 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:58.057251    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:57:58.971694 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:57:58.971653    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:58:08.855362 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:08.855316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:58:13.077826 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.077790    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" podUID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" containerName="main" containerID="cri-o://7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757" gracePeriod=45
Jul 06 01:58:13.323344 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.323320    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:58:13.341954 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.341883    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.341954 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.341942    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.342169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.341985    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tfmr\" (UniqueName: \"kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.342169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.342169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342069    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.342169 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342096    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir\") pod \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\" (UID: \"d1fa2b17-3d35-4172-949d-4c9ca8debb4a\") "
Jul 06 01:58:13.342332 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342253    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:13.342423 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342406    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.342477 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342413    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home" (OuterVolumeSpecName: "home") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:13.342477 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.342453    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:13.344420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.344388    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:58:13.344420 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.344402    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr" (OuterVolumeSpecName: "kube-api-access-6tfmr") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "kube-api-access-6tfmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:58:13.344569 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.344464    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm" (OuterVolumeSpecName: "dshm") pod "d1fa2b17-3d35-4172-949d-4c9ca8debb4a" (UID: "d1fa2b17-3d35-4172-949d-4c9ca8debb4a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:13.443114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.443077    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tfmr\" (UniqueName: \"kubernetes.io/projected/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-kube-api-access-6tfmr\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.443114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.443107    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.443114 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.443117    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.443357 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.443152    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.443357 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.443159    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fa2b17-3d35-4172-949d-4c9ca8debb4a-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:13.910665 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.910628    2567 generic.go:356] "Generic (PLEG): container finished" podID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" containerID="7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757" exitCode=0
Jul 06 01:58:13.910866 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.910672    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"
Jul 06 01:58:13.910866 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.910705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" event={"ID":"d1fa2b17-3d35-4172-949d-4c9ca8debb4a","Type":"ContainerDied","Data":"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"}
Jul 06 01:58:13.910866 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.910743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f" event={"ID":"d1fa2b17-3d35-4172-949d-4c9ca8debb4a","Type":"ContainerDied","Data":"ddec9ec9a234613d7d355802b8f96e17a33a91cc54a5fd370f560c63e3da1636"}
Jul 06 01:58:13.910866 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.910764    2567 scope.go:117] "RemoveContainer" containerID="7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"
Jul 06 01:58:13.919878 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.919858    2567 scope.go:117] "RemoveContainer" containerID="7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"
Jul 06 01:58:13.920109 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:58:13.920091    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757\": container with ID starting with 7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757 not found: ID does not exist" containerID="7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"
Jul 06 01:58:13.920187 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.920156    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757"} err="failed to get container status \"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757\": rpc error: code = NotFound desc = could not find container \"7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757\": container with ID starting with 7d4a73b145d1a1d7f82ed91276c7c0e1a51aed10c71b4f4d0a0bb2cbe38fd757 not found: ID does not exist"
Jul 06 01:58:13.932310 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.932288    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:58:13.935935 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.935913    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/sub-test-model-5afb9c-kserve-598877f595-qbk9f"]
Jul 06 01:58:13.992211 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:13.992182    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" podUID="9ed585ea-bd1b-492b-89b8-416b1219b96d" containerName="main" containerID="cri-o://6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db" gracePeriod=45
Jul 06 01:58:14.245054 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.245028    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:58:14.350551 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350516    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350713 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350562    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350713 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350584    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350713 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350630    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd5wc\" (UniqueName: \"kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350713 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350668    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350713 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350691    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir\") pod \"9ed585ea-bd1b-492b-89b8-416b1219b96d\" (UID: \"9ed585ea-bd1b-492b-89b8-416b1219b96d\") "
Jul 06 01:58:14.350981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350811    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:14.350981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350820    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home" (OuterVolumeSpecName: "home") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:14.350981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350945    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.350981 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.350961    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.351177 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.351002    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:14.353157 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.353100    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc" (OuterVolumeSpecName: "kube-api-access-rd5wc") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "kube-api-access-rd5wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:58:14.353157 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.353107    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm" (OuterVolumeSpecName: "dshm") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:58:14.353157 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.353138    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ed585ea-bd1b-492b-89b8-416b1219b96d" (UID: "9ed585ea-bd1b-492b-89b8-416b1219b96d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:58:14.451587 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.451517    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.451587 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.451541    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd5wc\" (UniqueName: \"kubernetes.io/projected/9ed585ea-bd1b-492b-89b8-416b1219b96d-kube-api-access-rd5wc\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.451587 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.451556    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ed585ea-bd1b-492b-89b8-416b1219b96d-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.451758 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.451592    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ed585ea-bd1b-492b-89b8-416b1219b96d-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:58:14.656267 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.656235    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" path="/var/lib/kubelet/pods/d1fa2b17-3d35-4172-949d-4c9ca8debb4a/volumes"
Jul 06 01:58:14.917756 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.917723    2567 generic.go:356] "Generic (PLEG): container finished" podID="9ed585ea-bd1b-492b-89b8-416b1219b96d" containerID="6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db" exitCode=0
Jul 06 01:58:14.917914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.917769    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"
Jul 06 01:58:14.917914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.917773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" event={"ID":"9ed585ea-bd1b-492b-89b8-416b1219b96d","Type":"ContainerDied","Data":"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"}
Jul 06 01:58:14.917914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.917803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q" event={"ID":"9ed585ea-bd1b-492b-89b8-416b1219b96d","Type":"ContainerDied","Data":"31c0705b39ebdd2593e39af827bb9a14d6bddb459f07d524ce72b5354f691863"}
Jul 06 01:58:14.917914 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.917818    2567 scope.go:117] "RemoveContainer" containerID="6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"
Jul 06 01:58:14.926465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.926447    2567 scope.go:117] "RemoveContainer" containerID="6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"
Jul 06 01:58:14.926751 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:58:14.926731    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db\": container with ID starting with 6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db not found: ID does not exist" containerID="6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"
Jul 06 01:58:14.926810 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.926758    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db"} err="failed to get container status \"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db\": rpc error: code = NotFound desc = could not find container \"6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db\": container with ID starting with 6925d149a5d92da62bf21d1862d9ab31615cb2b46495beef3629af5df16c67db not found: ID does not exist"
Jul 06 01:58:14.933848 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.933823    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:58:14.937013 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:14.936989    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/sub-test-model-ed9e8f-kserve-855d699765-whx8q"]
Jul 06 01:58:16.656225 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:16.656191    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed585ea-bd1b-492b-89b8-416b1219b96d" path="/var/lib/kubelet/pods/9ed585ea-bd1b-492b-89b8-416b1219b96d/volumes"
Jul 06 01:58:30.271886 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.271847    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:58:30.272461 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272444    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" containerName="main"
Jul 06 01:58:30.272518 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272465    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" containerName="main"
Jul 06 01:58:30.272518 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272513    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed585ea-bd1b-492b-89b8-416b1219b96d" containerName="main"
Jul 06 01:58:30.272597 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272522    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed585ea-bd1b-492b-89b8-416b1219b96d" containerName="main"
Jul 06 01:58:30.272633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272599    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed585ea-bd1b-492b-89b8-416b1219b96d" containerName="main"
Jul 06 01:58:30.272633 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.272613    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1fa2b17-3d35-4172-949d-4c9ca8debb4a" containerName="main"
Jul 06 01:58:30.277400 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.277381    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.280820 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.280797    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"rate-test-model-ed9e8f-kserve-self-signed-certs\""
Jul 06 01:58:30.281217 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.280915    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"default-dockercfg-rxhjd\""
Jul 06 01:58:30.281372 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.280973    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"openshift-service-ca.crt\""
Jul 06 01:58:30.281498 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.281023    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"kube-root-ca.crt\""
Jul 06 01:58:30.283374 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.283355    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:58:30.382844 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.382997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.382997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.382997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382895    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.382997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382967    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.382997 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.382989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwz9n\" (UniqueName: \"kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484213 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484345    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484380 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwz9n\" (UniqueName: \"kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484625    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484697 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484681    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.484802 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.484742    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.486706 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.486680    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.486902 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.486887    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.492829 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.492809    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwz9n\" (UniqueName: \"kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n\") pod \"rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") " pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.588631 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.588566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:30.722612 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.722588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:58:30.724936 ip-10-0-133-137 kubenswrapper[2567]: W0706 01:58:30.724911    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97c0eaad_722d_4843_8b48_c77a2c67019d.slice/crio-3eaef63a06556f9f59429b4369a2c60f81a4c43b2deaf2a262ee7dfbbb4cac77 WatchSource:0}: Error finding container 3eaef63a06556f9f59429b4369a2c60f81a4c43b2deaf2a262ee7dfbbb4cac77: Status 404 returned error can't find the container with id 3eaef63a06556f9f59429b4369a2c60f81a4c43b2deaf2a262ee7dfbbb4cac77
Jul 06 01:58:30.993780 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.993740    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" event={"ID":"97c0eaad-722d-4843-8b48-c77a2c67019d","Type":"ContainerStarted","Data":"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"}
Jul 06 01:58:30.993947 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:30.993789    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" event={"ID":"97c0eaad-722d-4843-8b48-c77a2c67019d","Type":"ContainerStarted","Data":"3eaef63a06556f9f59429b4369a2c60f81a4c43b2deaf2a262ee7dfbbb4cac77"}
Jul 06 01:58:31.013580 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:31.013529    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" podStartSLOduration=1.013515801 podStartE2EDuration="1.013515801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:58:31.011097171 +0000 UTC m=+2310.856246582" watchObservedRunningTime="2026-07-06 01:58:31.013515801 +0000 UTC m=+2310.858665213"
Jul 06 01:58:33.958104 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:33.958069    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:58:40.589090 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:40.589054    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:40.589619 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:40.589107    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:40.601821 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:40.601795    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:41.052371 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:41.052339    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:58:45.959261 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:58:45.959221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:59:06.260795 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:06.260763    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:59:10.673574 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:10.673543    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:59:12.455549 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.455512    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:59:12.455970 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.455755    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" podUID="487d1953-dd8e-4674-9127-362b513fc62c" containerName="maas-api" containerID="cri-o://b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa" gracePeriod=30
Jul 06 01:59:12.912416 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.912389    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:59:12.931335 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.931305    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbd2j\" (UniqueName: \"kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j\") pod \"487d1953-dd8e-4674-9127-362b513fc62c\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") "
Jul 06 01:59:12.931505 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.931352    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") pod \"487d1953-dd8e-4674-9127-362b513fc62c\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") "
Jul 06 01:59:12.931505 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.931423    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca\") pod \"487d1953-dd8e-4674-9127-362b513fc62c\" (UID: \"487d1953-dd8e-4674-9127-362b513fc62c\") "
Jul 06 01:59:12.931790 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.931759    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "487d1953-dd8e-4674-9127-362b513fc62c" (UID: "487d1953-dd8e-4674-9127-362b513fc62c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:59:12.933776 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.933743    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j" (OuterVolumeSpecName: "kube-api-access-rbd2j") pod "487d1953-dd8e-4674-9127-362b513fc62c" (UID: "487d1953-dd8e-4674-9127-362b513fc62c"). InnerVolumeSpecName "kube-api-access-rbd2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:59:12.933776 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:12.933758    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "487d1953-dd8e-4674-9127-362b513fc62c" (UID: "487d1953-dd8e-4674-9127-362b513fc62c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:59:13.032659 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.032578    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487d1953-dd8e-4674-9127-362b513fc62c-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:13.032659 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.032606    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbd2j\" (UniqueName: \"kubernetes.io/projected/487d1953-dd8e-4674-9127-362b513fc62c-kube-api-access-rbd2j\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:13.032659 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.032616    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487d1953-dd8e-4674-9127-362b513fc62c-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:13.195567 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.195534    2567 generic.go:356] "Generic (PLEG): container finished" podID="487d1953-dd8e-4674-9127-362b513fc62c" containerID="b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa" exitCode=0
Jul 06 01:59:13.195754 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.195579    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"
Jul 06 01:59:13.195754 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.195594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" event={"ID":"487d1953-dd8e-4674-9127-362b513fc62c","Type":"ContainerDied","Data":"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"}
Jul 06 01:59:13.195754 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.195647    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz" event={"ID":"487d1953-dd8e-4674-9127-362b513fc62c","Type":"ContainerDied","Data":"2d3fddfc78be708b6e130c156d72a552311886430e25659857190c48a426ed95"}
Jul 06 01:59:13.195754 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.195674    2567 scope.go:117] "RemoveContainer" containerID="b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"
Jul 06 01:59:13.205503 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.205485    2567 scope.go:117] "RemoveContainer" containerID="b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"
Jul 06 01:59:13.205803 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:59:13.205774    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa\": container with ID starting with b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa not found: ID does not exist" containerID="b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"
Jul 06 01:59:13.205904 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.205814    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa"} err="failed to get container status \"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa\": rpc error: code = NotFound desc = could not find container \"b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa\": container with ID starting with b53d91c446828818285eb0420e5df97f66f563e6f676ed582e0fa3e1b8cfa7fa not found: ID does not exist"
Jul 06 01:59:13.218258 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.218235    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:59:13.222253 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:13.222232    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5afb9c-7cf7587b47-rddnz"]
Jul 06 01:59:14.656335 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:14.656301    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="487d1953-dd8e-4674-9127-362b513fc62c" path="/var/lib/kubelet/pods/487d1953-dd8e-4674-9127-362b513fc62c/volumes"
Jul 06 01:59:20.559680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:20.559642    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 01:59:25.695801 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:25.695761    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" podUID="97c0eaad-722d-4843-8b48-c77a2c67019d" containerName="main" containerID="cri-o://b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad" gracePeriod=45
Jul 06 01:59:25.954623 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:25.954564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:59:26.024295 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024264    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024352    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwz9n\" (UniqueName: \"kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024374    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024465 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024415    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache\") pod \"97c0eaad-722d-4843-8b48-c77a2c67019d\" (UID: \"97c0eaad-722d-4843-8b48-c77a2c67019d\") "
Jul 06 01:59:26.024741 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024672    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home" (OuterVolumeSpecName: "home") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:59:26.024741 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024689    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:59:26.024741 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.024725    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache" (OuterVolumeSpecName: "model-cache") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:59:26.026554 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.026533    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm" (OuterVolumeSpecName: "dshm") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 01:59:26.026640 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.026580    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n" (OuterVolumeSpecName: "kube-api-access-vwz9n") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "kube-api-access-vwz9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:59:26.026680 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.026641    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "97c0eaad-722d-4843-8b48-c77a2c67019d" (UID: "97c0eaad-722d-4843-8b48-c77a2c67019d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:59:26.125029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.124999    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97c0eaad-722d-4843-8b48-c77a2c67019d-tls-certs\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.125029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.125025    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwz9n\" (UniqueName: \"kubernetes.io/projected/97c0eaad-722d-4843-8b48-c77a2c67019d-kube-api-access-vwz9n\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.125029 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.125034    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-home\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.125255 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.125043    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-tmp-dir\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.125255 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.125052    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-model-cache\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.125255 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.125061    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97c0eaad-722d-4843-8b48-c77a2c67019d-dshm\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:26.255656 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.255574    2567 generic.go:356] "Generic (PLEG): container finished" podID="97c0eaad-722d-4843-8b48-c77a2c67019d" containerID="b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad" exitCode=0
Jul 06 01:59:26.255656 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.255616    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"
Jul 06 01:59:26.255656 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.255647    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" event={"ID":"97c0eaad-722d-4843-8b48-c77a2c67019d","Type":"ContainerDied","Data":"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"}
Jul 06 01:59:26.255856 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.255683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb" event={"ID":"97c0eaad-722d-4843-8b48-c77a2c67019d","Type":"ContainerDied","Data":"3eaef63a06556f9f59429b4369a2c60f81a4c43b2deaf2a262ee7dfbbb4cac77"}
Jul 06 01:59:26.255856 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.255701    2567 scope.go:117] "RemoveContainer" containerID="b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"
Jul 06 01:59:26.264327 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.264307    2567 scope.go:117] "RemoveContainer" containerID="b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"
Jul 06 01:59:26.264581 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:59:26.264564    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad\": container with ID starting with b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad not found: ID does not exist" containerID="b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"
Jul 06 01:59:26.264645 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.264591    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad"} err="failed to get container status \"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad\": rpc error: code = NotFound desc = could not find container \"b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad\": container with ID starting with b263515f046a0f3255d796dcc6f6231c281ad66f41c23b3199c9c3f69ee1ecad not found: ID does not exist"
Jul 06 01:59:26.275276 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.275254    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:59:26.278872 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.278853    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/rate-test-model-ed9e8f-kserve-6db97455bc-tpdkb"]
Jul 06 01:59:26.658015 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:26.657984    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97c0eaad-722d-4843-8b48-c77a2c67019d" path="/var/lib/kubelet/pods/97c0eaad-722d-4843-8b48-c77a2c67019d/volumes"
Jul 06 01:59:55.413655 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.413619    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:59:55.414230 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.413936    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" podUID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" containerName="maas-api" containerID="cri-o://a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3" gracePeriod=30
Jul 06 01:59:55.662026 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.662003    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:59:55.756516 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.756445    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca\") pod \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") "
Jul 06 01:59:55.756516 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.756482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls\") pod \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") "
Jul 06 01:59:55.756516 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.756501    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdff5\" (UniqueName: \"kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5\") pod \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\" (UID: \"f3a10e6f-fedf-4081-b1ac-68d90c964c20\") "
Jul 06 01:59:55.756889 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.756861    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3a10e6f-fedf-4081-b1ac-68d90c964c20" (UID: "f3a10e6f-fedf-4081-b1ac-68d90c964c20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 01:59:55.758683 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.758666    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3a10e6f-fedf-4081-b1ac-68d90c964c20" (UID: "f3a10e6f-fedf-4081-b1ac-68d90c964c20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 01:59:55.758825 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.758803    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5" (OuterVolumeSpecName: "kube-api-access-hdff5") pod "f3a10e6f-fedf-4081-b1ac-68d90c964c20" (UID: "f3a10e6f-fedf-4081-b1ac-68d90c964c20"). InnerVolumeSpecName "kube-api-access-hdff5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 01:59:55.857451 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.857411    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a10e6f-fedf-4081-b1ac-68d90c964c20-openshift-service-ca\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:55.857451 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.857450    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a10e6f-fedf-4081-b1ac-68d90c964c20-maas-api-tls\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:55.857451 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:55.857460    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdff5\" (UniqueName: \"kubernetes.io/projected/f3a10e6f-fedf-4081-b1ac-68d90c964c20-kube-api-access-hdff5\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 01:59:56.392219 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.392183    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" containerID="a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3" exitCode=0
Jul 06 01:59:56.392391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.392240    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"
Jul 06 01:59:56.392391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.392273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" event={"ID":"f3a10e6f-fedf-4081-b1ac-68d90c964c20","Type":"ContainerDied","Data":"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"}
Jul 06 01:59:56.392391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.392324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc" event={"ID":"f3a10e6f-fedf-4081-b1ac-68d90c964c20","Type":"ContainerDied","Data":"6750f848e9b244299249aa689868cbb81a940c2fa217b5ff41842f28804e4d1e"}
Jul 06 01:59:56.392391 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.392350    2567 scope.go:117] "RemoveContainer" containerID="a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"
Jul 06 01:59:56.401823 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.401808    2567 scope.go:117] "RemoveContainer" containerID="a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"
Jul 06 01:59:56.402082 ip-10-0-133-137 kubenswrapper[2567]: E0706 01:59:56.402058    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3\": container with ID starting with a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3 not found: ID does not exist" containerID="a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"
Jul 06 01:59:56.402196 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.402095    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3"} err="failed to get container status \"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3\": rpc error: code = NotFound desc = could not find container \"a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3\": container with ID starting with a9197474a96f3cf21b17fd97d84058c036bb6de961891ee90806b25ac51b33b3 not found: ID does not exist"
Jul 06 01:59:56.412993 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.412969    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:59:56.416627 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.416606    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ed9e8f-645cbbcf5-c4ffc"]
Jul 06 01:59:56.656637 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:56.656549    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" path="/var/lib/kubelet/pods/f3a10e6f-fedf-4081-b1ac-68d90c964c20/volumes"
Jul 06 01:59:58.259752 ip-10-0-133-137 kubenswrapper[2567]: I0706 01:59:58.259718    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k5mcl"]
Jul 06 02:00:00.140246 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140212    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-l5kjd"]
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140543    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97c0eaad-722d-4843-8b48-c77a2c67019d" containerName="main"
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140554    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97c0eaad-722d-4843-8b48-c77a2c67019d" containerName="main"
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140568    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487d1953-dd8e-4674-9127-362b513fc62c" containerName="maas-api"
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140573    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="487d1953-dd8e-4674-9127-362b513fc62c" containerName="maas-api"
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140580    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" containerName="maas-api"
Jul 06 02:00:00.140618 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140585    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" containerName="maas-api"
Jul 06 02:00:00.140816 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140636    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="97c0eaad-722d-4843-8b48-c77a2c67019d" containerName="main"
Jul 06 02:00:00.140816 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140644    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a10e6f-fedf-4081-b1ac-68d90c964c20" containerName="maas-api"
Jul 06 02:00:00.140816 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.140651    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="487d1953-dd8e-4674-9127-362b513fc62c" containerName="maas-api"
Jul 06 02:00:00.145379 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.145357    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-l5kjd"]
Jul 06 02:00:00.145527 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.145444    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:00.148066 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.148047    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\""
Jul 06 02:00:00.194917 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.194887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssk7\" (UniqueName: \"kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7\") pod \"maas-api-key-cleanup-29721720-l5kjd\" (UID: \"bdc02172-6e5f-45ea-88eb-59b291c006e6\") " pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:00.295729 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.295695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssk7\" (UniqueName: \"kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7\") pod \"maas-api-key-cleanup-29721720-l5kjd\" (UID: \"bdc02172-6e5f-45ea-88eb-59b291c006e6\") " pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:00.304323 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.304300    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssk7\" (UniqueName: \"kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7\") pod \"maas-api-key-cleanup-29721720-l5kjd\" (UID: \"bdc02172-6e5f-45ea-88eb-59b291c006e6\") " pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:00.455808 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.455729    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:00.591558 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.591537    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-l5kjd"]
Jul 06 02:00:00.593108 ip-10-0-133-137 kubenswrapper[2567]: W0706 02:00:00.593082    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc02172_6e5f_45ea_88eb_59b291c006e6.slice/crio-1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c WatchSource:0}: Error finding container 1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c: Status 404 returned error can't find the container with id 1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c
Jul 06 02:00:00.594979 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.594960    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 02:00:00.750886 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.750808    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 02:00:00.764672 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:00.764650    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 02:00:01.417752 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:01.417720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd" event={"ID":"bdc02172-6e5f-45ea-88eb-59b291c006e6","Type":"ContainerStarted","Data":"1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c"}
Jul 06 02:00:02.177209 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:02.177174    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff6f8b7f4-6xj7v_cbb15602-250e-4da9-9ab1-bcb7a34f60f4/authorino/0.log"
Jul 06 02:00:04.433798 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:04.433762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd" event={"ID":"bdc02172-6e5f-45ea-88eb-59b291c006e6","Type":"ContainerStarted","Data":"575361e350cb889680e3488a860a2f11931d75d830bc025d94091339b427a874"}
Jul 06 02:00:04.448473 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:04.448427    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd" podStartSLOduration=1.4191318339999999 podStartE2EDuration="4.448411475s" totalImagesPullingTime="3.029279641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:00:04.446663865 +0000 UTC m=+2404.291813297" watchObservedRunningTime="2026-07-06 02:00:04.448411475 +0000 UTC m=+2404.293560887"
Jul 06 02:00:05.439841 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:05.439803    2567 generic.go:356] "Generic (PLEG): container finished" podID="bdc02172-6e5f-45ea-88eb-59b291c006e6" containerID="575361e350cb889680e3488a860a2f11931d75d830bc025d94091339b427a874" exitCode=0
Jul 06 02:00:05.440225 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:05.439890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd" event={"ID":"bdc02172-6e5f-45ea-88eb-59b291c006e6","Type":"ContainerDied","Data":"575361e350cb889680e3488a860a2f11931d75d830bc025d94091339b427a874"}
Jul 06 02:00:06.578983 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:06.578955    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:06.648540 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:06.648511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nssk7\" (UniqueName: \"kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7\") pod \"bdc02172-6e5f-45ea-88eb-59b291c006e6\" (UID: \"bdc02172-6e5f-45ea-88eb-59b291c006e6\") "
Jul 06 02:00:06.650676 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:06.650654    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7" (OuterVolumeSpecName: "kube-api-access-nssk7") pod "bdc02172-6e5f-45ea-88eb-59b291c006e6" (UID: "bdc02172-6e5f-45ea-88eb-59b291c006e6"). InnerVolumeSpecName "kube-api-access-nssk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 02:00:06.749627 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:06.749555    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nssk7\" (UniqueName: \"kubernetes.io/projected/bdc02172-6e5f-45ea-88eb-59b291c006e6-kube-api-access-nssk7\") on node \"ip-10-0-133-137.ec2.internal\" DevicePath \"\""
Jul 06 02:00:07.183757 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.183722    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8xl24_c0eddcbf-0df4-43ce-927e-5e694d38f185/manager/0.log"
Jul 06 02:00:07.301515 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.301490    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fj8fk_25006f46-ee92-4f37-b393-5bad384d3483/manager/0.log"
Jul 06 02:00:07.450139 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.450022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd" event={"ID":"bdc02172-6e5f-45ea-88eb-59b291c006e6","Type":"ContainerDied","Data":"1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c"}
Jul 06 02:00:07.450139 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.450077    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1806b5662ac3ea047616cadc75afb28c533d0506020e8405380bd51b6cd6ff2c"
Jul 06 02:00:07.450139 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.450038    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-l5kjd"
Jul 06 02:00:07.758512 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:07.758412    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721720-l5kjd_bdc02172-6e5f-45ea-88eb-59b291c006e6/cleanup/0.log"
Jul 06 02:00:08.010343 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:08.010270    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pd7kl_b7982eb9-e361-47df-ba5f-4ab83bb12257/server/0.log"
Jul 06 02:00:08.137085 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:08.137054    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2mqd8_37ce2ac4-c403-44f2-9760-3ceaacf3584d/manager/0.log"
Jul 06 02:00:08.367373 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:08.367352    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-ccmkr_ae8b3a91-62b7-46b5-928d-88b105a0b778/copy-manifests/0.log"
Jul 06 02:00:08.385263 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:08.385230    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-ccmkr_ae8b3a91-62b7-46b5-928d-88b105a0b778/manager/0.log"
Jul 06 02:00:09.885059 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:09.885027    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff6f8b7f4-6xj7v_cbb15602-250e-4da9-9ab1-bcb7a34f60f4/authorino/0.log"
Jul 06 02:00:10.000997 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.000971    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fsf4h_37040ac5-5b89-458e-a170-d8c2ff75e594/manager/0.log"
Jul 06 02:00:10.109665 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.109623    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9zmvj_62188051-0e95-44e6-ad1d-138fa92f4399/manager/0.log"
Jul 06 02:00:10.215852 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.215780    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xlghp_d1411357-0205-495f-8ac2-28472483e807/kuadrant-console-plugin/0.log"
Jul 06 02:00:10.336993 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.336962    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xnrhn_141cee8e-3fc7-4ba4-ba1b-67040c43c3b8/registry-server/0.log"
Jul 06 02:00:10.572351 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.572274    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k5mcl_c61e84f1-a5da-461c-87b8-e4b4dd5d8d76/limitador/0.log"
Jul 06 02:00:10.699878 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:10.699848    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gnnxr_671b3910-20c4-4407-b87b-59ee49dcf1e9/manager/0.log"
Jul 06 02:00:11.362068 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:11.362037    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-mzzh5_dd0b9b58-44f5-4432-ad90-2afc08769bed/kube-auth-proxy/0.log"
Jul 06 02:00:12.461925 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:12.461896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db8hcjv_67e24451-d74e-4487-ae42-243fe9dd2c73/main/0.log"
Jul 06 02:00:12.686062 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:12.686029    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-4nww6_c526d51f-6859-4b53-b04d-c1ff3019ab42/main/0.log"
Jul 06 02:00:16.671972 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.671937    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hqwmc/must-gather-pzkpf"]
Jul 06 02:00:16.672388 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.672269    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc02172-6e5f-45ea-88eb-59b291c006e6" containerName="cleanup"
Jul 06 02:00:16.672388 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.672280    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc02172-6e5f-45ea-88eb-59b291c006e6" containerName="cleanup"
Jul 06 02:00:16.672388 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.672340    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc02172-6e5f-45ea-88eb-59b291c006e6" containerName="cleanup"
Jul 06 02:00:16.675545 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.675525    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.678146 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.678102    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqwmc\"/\"openshift-service-ca.crt\""
Jul 06 02:00:16.678382 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.678352    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqwmc\"/\"kube-root-ca.crt\""
Jul 06 02:00:16.678482 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.678387    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hqwmc\"/\"default-dockercfg-5sss8\""
Jul 06 02:00:16.683716 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.683696    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/must-gather-pzkpf"]
Jul 06 02:00:16.831250 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.831207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4jl\" (UniqueName: \"kubernetes.io/projected/30cd3714-c97b-4623-9db3-e1cc5843f726-kube-api-access-zp4jl\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.831441 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.831285    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/30cd3714-c97b-4623-9db3-e1cc5843f726-must-gather-output\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.932312 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.932225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4jl\" (UniqueName: \"kubernetes.io/projected/30cd3714-c97b-4623-9db3-e1cc5843f726-kube-api-access-zp4jl\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.932312 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.932278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/30cd3714-c97b-4623-9db3-e1cc5843f726-must-gather-output\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.932670 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.932647    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/30cd3714-c97b-4623-9db3-e1cc5843f726-must-gather-output\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.940073 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.940049    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4jl\" (UniqueName: \"kubernetes.io/projected/30cd3714-c97b-4623-9db3-e1cc5843f726-kube-api-access-zp4jl\") pod \"must-gather-pzkpf\" (UID: \"30cd3714-c97b-4623-9db3-e1cc5843f726\") " pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:16.985306 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:16.985277    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/must-gather-pzkpf"
Jul 06 02:00:17.117971 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:17.117939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/must-gather-pzkpf"]
Jul 06 02:00:17.120892 ip-10-0-133-137 kubenswrapper[2567]: W0706 02:00:17.120857    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30cd3714_c97b_4623_9db3_e1cc5843f726.slice/crio-5ded1d84f2e6429aa5e935caa25b25832bae936ce07d59aa6f8228ca12c4fe0a WatchSource:0}: Error finding container 5ded1d84f2e6429aa5e935caa25b25832bae936ce07d59aa6f8228ca12c4fe0a: Status 404 returned error can't find the container with id 5ded1d84f2e6429aa5e935caa25b25832bae936ce07d59aa6f8228ca12c4fe0a
Jul 06 02:00:17.500479 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:17.500432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/must-gather-pzkpf" event={"ID":"30cd3714-c97b-4623-9db3-e1cc5843f726","Type":"ContainerStarted","Data":"5ded1d84f2e6429aa5e935caa25b25832bae936ce07d59aa6f8228ca12c4fe0a"}
Jul 06 02:00:18.510062 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:18.510024    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/must-gather-pzkpf" event={"ID":"30cd3714-c97b-4623-9db3-e1cc5843f726","Type":"ContainerStarted","Data":"cc096a8ed9d464bc4708bbd84302f9bbd2ef009264c08e0d67d6f26b2f2eabd5"}
Jul 06 02:00:18.510566 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:18.510076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/must-gather-pzkpf" event={"ID":"30cd3714-c97b-4623-9db3-e1cc5843f726","Type":"ContainerStarted","Data":"f29c4958fcd06f96dd7a924a5f489df75ad7797dd687b0a616e846f46f148842"}
Jul 06 02:00:18.524914 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:18.524853    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hqwmc/must-gather-pzkpf" podStartSLOduration=1.536825802 podStartE2EDuration="2.524831392s" totalImagesPullingTime="988.00559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:00:18.523838493 +0000 UTC m=+2418.368987906" watchObservedRunningTime="2026-07-06 02:00:18.524831392 +0000 UTC m=+2418.369980805"
Jul 06 02:00:19.621178 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:19.621146    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gc54p_0c277b2c-503f-48a2-9429-6ad2c2094008/global-pull-secret-syncer/0.log"
Jul 06 02:00:19.789491 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:19.789457    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-htk6z_53f29e00-0b09-48e9-8218-e1885f613b85/konnectivity-agent/0.log"
Jul 06 02:00:19.842640 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:19.842616    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-137.ec2.internal_c581f67582f2b7493459f83b250c1488/haproxy/0.log"
Jul 06 02:00:24.439203 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.438667    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff6f8b7f4-6xj7v_cbb15602-250e-4da9-9ab1-bcb7a34f60f4/authorino/0.log"
Jul 06 02:00:24.472419 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.472381    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fsf4h_37040ac5-5b89-458e-a170-d8c2ff75e594/manager/0.log"
Jul 06 02:00:24.517941 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.517896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9zmvj_62188051-0e95-44e6-ad1d-138fa92f4399/manager/0.log"
Jul 06 02:00:24.547287 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.547255    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xlghp_d1411357-0205-495f-8ac2-28472483e807/kuadrant-console-plugin/0.log"
Jul 06 02:00:24.599500 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.599449    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xnrhn_141cee8e-3fc7-4ba4-ba1b-67040c43c3b8/registry-server/0.log"
Jul 06 02:00:24.768958 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.768859    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k5mcl_c61e84f1-a5da-461c-87b8-e4b4dd5d8d76/limitador/0.log"
Jul 06 02:00:24.870799 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:24.869850    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gnnxr_671b3910-20c4-4407-b87b-59ee49dcf1e9/manager/0.log"
Jul 06 02:00:27.062984 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:27.062946    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n52xs_97ad9e35-0895-466d-bfe3-2ba723e75bad/node-exporter/0.log"
Jul 06 02:00:27.090055 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:27.090030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n52xs_97ad9e35-0895-466d-bfe3-2ba723e75bad/kube-rbac-proxy/0.log"
Jul 06 02:00:27.118964 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:27.118939    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n52xs_97ad9e35-0895-466d-bfe3-2ba723e75bad/init-textfile/0.log"
Jul 06 02:00:28.951986 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:28.951951    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-lkvb8_5d8e6636-d0a2-4a64-9f85-71c81dc678e4/networking-console-plugin/0.log"
Jul 06 02:00:30.459838 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.459801    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"]
Jul 06 02:00:30.465772 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.465748    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.473102 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.473067    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"]
Jul 06 02:00:30.567388 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.567342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-proc\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.567621 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.567452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-lib-modules\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.567621 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.567499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stxz5\" (UniqueName: \"kubernetes.io/projected/6d292ff0-31a1-4b43-94e6-bf671c047fa6-kube-api-access-stxz5\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.567621 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.567531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-sys\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.567621 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.567556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-podres\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668556 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-lib-modules\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668556 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stxz5\" (UniqueName: \"kubernetes.io/projected/6d292ff0-31a1-4b43-94e6-bf671c047fa6-kube-api-access-stxz5\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-sys\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-podres\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668700    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-lib-modules\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-proc\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668766    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-sys\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.668822 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668798    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-proc\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.669164 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.668879    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d292ff0-31a1-4b43-94e6-bf671c047fa6-podres\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.677988 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.677952    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stxz5\" (UniqueName: \"kubernetes.io/projected/6d292ff0-31a1-4b43-94e6-bf671c047fa6-kube-api-access-stxz5\") pod \"perf-node-gather-daemonset-rjsx9\" (UID: \"6d292ff0-31a1-4b43-94e6-bf671c047fa6\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.779264 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.779188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:30.961175 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:30.961139    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"]
Jul 06 02:00:30.964147 ip-10-0-133-137 kubenswrapper[2567]: W0706 02:00:30.964098    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6d292ff0_31a1_4b43_94e6_bf671c047fa6.slice/crio-79ce4d704779153a8386d656a14b37d8b137d464478335031219760a8605d930 WatchSource:0}: Error finding container 79ce4d704779153a8386d656a14b37d8b137d464478335031219760a8605d930: Status 404 returned error can't find the container with id 79ce4d704779153a8386d656a14b37d8b137d464478335031219760a8605d930
Jul 06 02:00:31.600988 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:31.600944    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9" event={"ID":"6d292ff0-31a1-4b43-94e6-bf671c047fa6","Type":"ContainerStarted","Data":"67684dcb1dd29e9c436ab7c091c1a3ea845e5b1baf5d5039d65b3ef53688e429"}
Jul 06 02:00:31.601502 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:31.601014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9" event={"ID":"6d292ff0-31a1-4b43-94e6-bf671c047fa6","Type":"ContainerStarted","Data":"79ce4d704779153a8386d656a14b37d8b137d464478335031219760a8605d930"}
Jul 06 02:00:31.601827 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:31.601799    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:31.620429 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:31.620371    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9" podStartSLOduration=1.620350824 podStartE2EDuration="1.620350824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:00:31.617524418 +0000 UTC m=+2431.462673832" watchObservedRunningTime="2026-07-06 02:00:31.620350824 +0000 UTC m=+2431.465500237"
Jul 06 02:00:32.082244 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:32.082208    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cs8_1c306948-74cd-4b86-9383-c2687d0503c5/dns/0.log"
Jul 06 02:00:32.102710 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:32.102652    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cs8_1c306948-74cd-4b86-9383-c2687d0503c5/kube-rbac-proxy/0.log"
Jul 06 02:00:32.169740 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:32.169711    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s6r8x_e0122480-e48e-449b-b4eb-2f78298552b4/dns-node-resolver/0.log"
Jul 06 02:00:32.691331 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:32.691299    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-878457b9b-mn748_8d16168d-dc9a-46cb-86ff-006409ed781b/registry/0.log"
Jul 06 02:00:32.713084 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:32.713050    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26d9w_1d794170-f755-4376-842b-17583c87c9c5/node-ca/0.log"
Jul 06 02:00:33.809169 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:33.809113    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-mzzh5_dd0b9b58-44f5-4432-ad90-2afc08769bed/kube-auth-proxy/0.log"
Jul 06 02:00:34.629035 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:34.629003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ctfzm_6c131c57-4eb6-45fd-9dd8-76c09f647891/serve-healthcheck-canary/0.log"
Jul 06 02:00:35.241705 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:35.241665    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-twncn_992bbf59-52ef-431f-b678-552c6ef05c3b/kube-rbac-proxy/0.log"
Jul 06 02:00:35.261253 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:35.261226    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-twncn_992bbf59-52ef-431f-b678-552c6ef05c3b/exporter/0.log"
Jul 06 02:00:35.282851 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:35.282823    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-twncn_992bbf59-52ef-431f-b678-552c6ef05c3b/extractor/0.log"
Jul 06 02:00:37.315743 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:37.315715    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8xl24_c0eddcbf-0df4-43ce-927e-5e694d38f185/manager/0.log"
Jul 06 02:00:37.566048 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:37.565935    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fj8fk_25006f46-ee92-4f37-b393-5bad384d3483/manager/0.log"
Jul 06 02:00:37.645573 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:37.645543    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721720-l5kjd_bdc02172-6e5f-45ea-88eb-59b291c006e6/cleanup/0.log"
Jul 06 02:00:37.764942 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:37.764916    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pd7kl_b7982eb9-e361-47df-ba5f-4ab83bb12257/server/0.log"
Jul 06 02:00:37.878870 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:37.878836    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2mqd8_37ce2ac4-c403-44f2-9760-3ceaacf3584d/manager/0.log"
Jul 06 02:00:38.039395 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:38.039359    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-ccmkr_ae8b3a91-62b7-46b5-928d-88b105a0b778/manager/0.log"
Jul 06 02:00:38.056998 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:38.056976    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-ccmkr_ae8b3a91-62b7-46b5-928d-88b105a0b778/copy-manifests/0.log"
Jul 06 02:00:38.626074 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:38.626045    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-rjsx9"
Jul 06 02:00:40.221996 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:40.221971    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-mzzh5_dd0b9b58-44f5-4432-ad90-2afc08769bed/kube-auth-proxy/0.log"
Jul 06 02:00:41.847628 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:41.847593    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-mzzh5_dd0b9b58-44f5-4432-ad90-2afc08769bed/kube-auth-proxy/0.log"
Jul 06 02:00:46.010655 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.010615    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/kube-multus-additional-cni-plugins/0.log"
Jul 06 02:00:46.030173 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.030147    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/egress-router-binary-copy/0.log"
Jul 06 02:00:46.048541 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.048524    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/cni-plugins/0.log"
Jul 06 02:00:46.066741 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.066723    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/bond-cni-plugin/0.log"
Jul 06 02:00:46.085803 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.085782    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/routeoverride-cni/0.log"
Jul 06 02:00:46.105692 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.105670    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/whereabouts-cni-bincopy/0.log"
Jul 06 02:00:46.126092 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.126067    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5smz7_f3802e17-f464-4bd6-b7ea-7ebfa91b5cf8/whereabouts-cni/0.log"
Jul 06 02:00:46.345991 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.345901    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tcksn_8373e415-2493-428c-b644-6d236f61f4ee/kube-multus/0.log"
Jul 06 02:00:46.452604 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.452570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ktvvs_7bc9f526-fc74-4399-95db-9cd060968c31/network-metrics-daemon/0.log"
Jul 06 02:00:46.468570 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:46.468547    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ktvvs_7bc9f526-fc74-4399-95db-9cd060968c31/kube-rbac-proxy/0.log"
Jul 06 02:00:47.468988 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.468956    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-controller/0.log"
Jul 06 02:00:47.486647 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.486606    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/1.log"
Jul 06 02:00:47.486647 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.486622    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovn-acl-logging/0.log"
Jul 06 02:00:47.507584 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.507550    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/kube-rbac-proxy-node/0.log"
Jul 06 02:00:47.527022 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.526998    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/kube-rbac-proxy-ovn-metrics/0.log"
Jul 06 02:00:47.542369 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.542346    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/northd/0.log"
Jul 06 02:00:47.561829 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.561807    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/nbdb/0.log"
Jul 06 02:00:47.582035 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.582015    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/sbdb/0.log"
Jul 06 02:00:47.666445 ip-10-0-133-137 kubenswrapper[2567]: I0706 02:00:47.666415    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgsgd_60d676d2-25be-4f71-b759-f3e95c48d690/ovnkube-controller/0.log"