Jul 28 12:51:26.558067 ip-10-0-132-196 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 12:51:26.954957 ip-10-0-132-196 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:51:26.954957 ip-10-0-132-196 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 12:51:26.954957 ip-10-0-132-196 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:51:26.954957 ip-10-0-132-196 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 12:51:26.954957 ip-10-0-132-196 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:51:26.955921 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.955791    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 12:51:26.962071 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962051    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.962071 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962071    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962075    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962080    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962083    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962086    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962089    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962092    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962095    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962103    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962106    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962109    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962112    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962114    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962117    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962119    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962122    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962124    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962127    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962130    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962132    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.962149 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962134    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962137    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962139    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962143    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962145    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962147    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962150    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962154    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962156    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962159    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962162    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962165    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962167    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962169    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962172    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962174    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962177    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962179    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962182    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962185    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.962617 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962187    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962190    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962192    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962194    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962197    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962199    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962203    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962205    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962208    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962210    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962213    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962215    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962217    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962223    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962227    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962230    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962233    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962236    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962239    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.963125 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962243    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962246    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962248    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962251    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962253    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962256    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962258    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962260    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962263    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962265    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962268    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962270    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962273    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962276    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962278    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962281    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962284    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962286    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962288    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962291    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.963594 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962295    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962298    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962300    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962303    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962306    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962308    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962312    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962316    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962319    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962322    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962325    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962328    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962330    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962333    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962336    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962339    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962341    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962344    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.964102 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.962346    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.965450 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965436    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.965450 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965451    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965454    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965458    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965460    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965463    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965465    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965470    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965472    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965475    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965477    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965481    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965485    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965488    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965491    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965494    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965496    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965499    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965501    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965504    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965507    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.965514 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965509    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965511    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965514    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965517    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965519    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965522    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965525    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965528    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965531    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965534    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965538    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965541    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965543    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965546    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965548    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965551    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965553    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965555    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965558    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965561    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.966072 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965563    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965565    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965568    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965570    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965572    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965575    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965578    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965580    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965583    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965585    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965588    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965590    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965593    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965595    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965600    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965602    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965604    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965607    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965610    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965612    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.966574 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965615    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965618    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965620    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965623    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965626    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965628    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965630    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965633    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965635    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965639    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965642    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965644    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965647    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965649    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965652    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965655    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965657    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965660    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965664    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965666    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.967173 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965668    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965671    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965673    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965676    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965678    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965681    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965685    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965688    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965690    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965693    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965695    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965698    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965700    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965702    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965705    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965707    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965710    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.965712    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966124    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.967664 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966129    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966132    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966135    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966137    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966140    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966144    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966147    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966150    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966152    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966155    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966158    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966160    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966163    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966166    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966168    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966171    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966173    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966176    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966178    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966181    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.968165 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966183    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966186    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966188    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966191    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966193    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966195    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966198    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966200    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966203    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966205    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966208    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966210    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966212    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966215    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966219    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966223    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966225    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966228    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966231    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.968671 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966235    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966238    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966242    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966245    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966247    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966251    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966253    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966256    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966259    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966262    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966265    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966267    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966269    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966271    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966274    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966276    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966279    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966281    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966284    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966288    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.969169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966291    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966294    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966296    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966299    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966301    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966304    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966306    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966309    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966312    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966314    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966317    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966319    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966322    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966324    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966327    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966330    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966332    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966335    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966337    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.969655 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966339    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966342    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966344    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966347    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966349    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966352    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966354    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966356    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966358    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966361    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966363    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966365    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966368    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966370    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966372    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966375    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966377    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966380    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966382    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.966385    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.970133 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966461    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966472    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966478    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966482    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966487    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966490    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966495    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966500    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966504    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966507    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966510    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966513    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966516    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966519    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966523    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966526    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966529    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966531    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966535    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966538    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966541    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966544    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966548    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966551    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 12:51:26.970659 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966554    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966557    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966560    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966563    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966565    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966568    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966571    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966574    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966578    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966582    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966585    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966588    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966592    2562 flags.go:64] FLAG: --enable-server="true"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966595    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966598    2562 flags.go:64] FLAG: --event-burst="100"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966603    2562 flags.go:64] FLAG: --event-qps="50"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966606    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966610    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966613    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966616    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966619    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966622    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966625    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966628    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966631    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 12:51:26.971293 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966634    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966637    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966640    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966642    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966645    2562 flags.go:64] FLAG: --feature-gates=""
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966649    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966652    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966655    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966659    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966661    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966664    2562 flags.go:64] FLAG: --help="false"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966667    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-196.ec2.internal"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966670    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966673    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966676    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966679    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966684    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966687    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966690    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966692    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966695    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966698    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966704    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966708    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 28 12:51:26.971936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966711    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966713    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966716    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966719    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966722    2562 flags.go:64] FLAG: --lock-file=""
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966725    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966727    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966730    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966749    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966752    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966755    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966758    2562 flags.go:64] FLAG: --logging-format="text"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966761    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966764    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966767    2562 flags.go:64] FLAG: --manifest-url=""
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966771    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966775    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966778    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966782    2562 flags.go:64] FLAG: --max-pods="110"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966785    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966788    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966791    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966794    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966797    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966802    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 12:51:26.972524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966805    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966812    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966815    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966818    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966821    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966825    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966830    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966833    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966836    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966840    2562 flags.go:64] FLAG: --port="10250"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966843    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966846    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aac6d812e8b7db7d"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966849    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966851    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966854    2562 flags.go:64] FLAG: --register-node="true"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966857    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966860    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966863    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966866    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966869    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966872    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966875    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966878    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966881    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966884    2562 flags.go:64] FLAG: --runonce="false"
Jul 28 12:51:26.973250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966886    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966889    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966892    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966895    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966898    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966900    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966905    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966908    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966911    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966914    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966917    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966919    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966922    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966927    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966930    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966935    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966938    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966941    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966945    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966947    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966950    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966953    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966956    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966958    2562 flags.go:64] FLAG: --v="2"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966962    2562 flags.go:64] FLAG: --version="false"
Jul 28 12:51:26.973904 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966966    2562 flags.go:64] FLAG: --vmodule=""
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966970    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.966973    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967124    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967128    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967131    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967134    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967138    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967140    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967144    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967146    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967149    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967151    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967155    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967158    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967160    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967162    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967165    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967167    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967170    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.974727 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967174    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967177    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967179    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967183    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967185    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967188    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967191    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967193    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967196    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967198    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967202    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967206    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967210    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967224    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967228    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967231    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967234    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967237    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967240    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.975467 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967243    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967246    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967249    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967251    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967254    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967257    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967260    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967263    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967266    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967269    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967272    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967274    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967276    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967280    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967283    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967285    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967288    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967291    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967293    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967295    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.976148 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967298    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967300    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967303    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967305    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967308    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967310    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967313    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967315    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967318    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967320    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967322    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967325    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967327    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967330    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967332    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967335    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967337    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967340    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967344    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967346    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.976635 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967348    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967351    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967353    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967356    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967358    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967362    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967364    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967367    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967369    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967372    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967375    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967377    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967379    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967382    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967384    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967387    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967389    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967391    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967394    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967397    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.977146 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967399    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967402    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.967404    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.968165    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.976761    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.976778    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.976805    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.976811    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976874    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976879    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976884    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976888    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976891    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976895    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.977640 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976898    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976901    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976903    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976906    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976909    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976912    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976915    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976917    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976920    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976922    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976924    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976927    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976929    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976932    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976934    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976937    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976939    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976942    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976944    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976947    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.978015 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976949    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976953    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976955    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976957    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976960    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976962    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976965    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976967    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976970    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976972    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976974    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976977    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976979    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976982    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976985    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976987    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976990    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976993    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976995    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.976998    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.978503 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977000    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977003    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977005    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977008    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977011    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977013    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977016    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977018    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977020    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977023    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977026    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977028    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977031    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977033    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977037    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977040    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977044    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977046    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977049    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.979022 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977051    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977054    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977056    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977059    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977061    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977063    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977066    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977069    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977071    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977074    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977076    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977079    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977081    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977084    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977087    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977090    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977092    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977095    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977098    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.979488 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977101    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977103    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977105    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977108    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977111    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977114    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977117    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977119    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977122    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977124    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977127    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977129    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977132    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977134    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977137    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.980027 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.977142    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977254    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977259    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977262    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977266    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977270    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977273    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977276    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977279    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977281    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977284    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977286    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977289    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977292    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977295    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977297    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977300    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977303    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977306    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:51:26.980393 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977308    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977311    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977313    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977316    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977318    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977321    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977323    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977325    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977328    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977332    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977335    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977338    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977341    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977344    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977347    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977349    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977352    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977355    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977357    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:51:26.980944 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977360    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977362    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977365    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977368    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977370    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977373    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977375    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977377    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977381    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977383    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977385    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977388    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977390    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977392    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977395    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977397    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977399    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977402    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977404    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977406    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:51:26.981407 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977409    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977412    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977414    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977417    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977419    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977422    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977424    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977426    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977429    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977431    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977434    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977437    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977439    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977441    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977444    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977446    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977449    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977451    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977453    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977456    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:51:26.981904 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977458    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977462    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977464    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977467    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977469    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977474    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977476    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977479    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977481    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977483    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977486    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977488    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977491    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977493    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977495    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977498    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977502    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977505    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977507    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977510    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:51:26.982382 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977512    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:26.977515    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.977521    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.977635    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.979697    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.981019    2562 server.go:1019] "Starting client certificate rotation"
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.981128    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:51:26.982882 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.981168    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:51:26.984057 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.984043    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:51:26.985308 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.985289    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:51:26.996507 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.996484    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 28 12:51:26.998902 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.998886    2562 log.go:25] "Validated CRI v1 image API"
Jul 28 12:51:26.999857 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:26.999842    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 12:51:27.004422 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.004392    2562 fs.go:135] Filesystem UUIDs: map[49ffe9ce-a368-4bf7-9ec0-e57bf3ebd5d9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e99dde49-8d7d-4ec3-83bd-e0756ec546b6:/dev/nvme0n1p3]
Jul 28 12:51:27.004508 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.004418    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 12:51:27.011765 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.011629    2562 manager.go:216] Machine: {Timestamp:2026-07-28 12:51:27.010290324 +0000 UTC m=+0.327309310 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101974 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec245da71156573898454e843766c835 SystemUUID:ec245da7-1156-5738-9845-4e843766c835 BootID:a82d6f9e-58ed-4a90-bd96-97ea99070636 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:98:74:76:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:98:74:76:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:df:9b:50:e9:15 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 12:51:27.011765 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.011751    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 12:51:27.011899 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.011842    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 12:51:27.014591 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.014551    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 12:51:27.014760 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.014594    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 12:51:27.015780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.015768    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 12:51:27.015780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.015781    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 12:51:27.015853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.015789    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:51:27.016529 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.016518    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:51:27.017271 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.017259    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 12:51:27.017392 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.017374    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:51:27.018946 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.018934    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:51:27.019057 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019048    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 12:51:27.019094 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019078    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 12:51:27.019094 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019087    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 12:51:27.019167 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019102    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 12:51:27.019167 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019120    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 28 12:51:27.019167 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019129    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 12:51:27.019536 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019522    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:51:27.019582 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.019546    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:51:27.021826 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.021809    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 12:51:27.023584 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.023569    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 12:51:27.023644 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.023601    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 12:51:27.025200 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025188    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 12:51:27.025238 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025206    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 12:51:27.025238 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025214    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 12:51:27.025238 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025223    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 12:51:27.025238 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025234    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 12:51:27.025238 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025240    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025248    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025254    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025266    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025271    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025280    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 12:51:27.025417 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.025285    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 12:51:27.026490 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.026476    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 12:51:27.026490 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.026487    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 12:51:27.029889 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.029867    2562 server.go:1268] "Started kubelet"
Jul 28 12:51:27.029984 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.029915    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 12:51:27.030049 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.029995    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 12:51:27.030102 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.030081    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 12:51:27.030806 ip-10-0-132-196 systemd[1]: Started Kubernetes Kubelet.
Jul 28 12:51:27.031878 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.031787    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 12:51:27.032056 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.032044    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 12:51:27.034300 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.034275    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 12:51:27.034453 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.034299    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 12:51:27.034453 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.034297    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 12:51:27.035596 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.035569    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 12:51:27.038094 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038075    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 12:51:27.038094 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038078    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 12:51:27.038689 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.038174    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.038830 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038769    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 12:51:27.038830 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038782    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 12:51:27.038982 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038894    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 12:51:27.038982 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038905    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 12:51:27.038982 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.038938    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.041944    2562 factory.go:55] Registering systemd factory
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.041982    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.042512    2562 factory.go:153] Registering CRI-O factory
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.042529    2562 factory.go:223] Registration of the crio container factory successfully
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.042601    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.042625    2562 factory.go:103] Registering Raw factory
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.042643    2562 manager.go:1196] Started watching for new ooms in manager
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.043233    2562 manager.go:349] Starting recovery of all containers
Jul 28 12:51:27.045300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.044489    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wrppt"
Jul 28 12:51:27.048541 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.048497    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 12:51:27.049147 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.047854    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-196.ec2.internal.18c67521082261b7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-196.ec2.internal,UID:ip-10-0-132-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-196.ec2.internal,},FirstTimestamp:2026-07-28 12:51:27.029821879 +0000 UTC m=+0.346840861,LastTimestamp:2026-07-28 12:51:27.029821879 +0000 UTC m=+0.346840861,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-196.ec2.internal,}"
Jul 28 12:51:27.050931 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.050874    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 12:51:27.053448 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.053425    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wrppt"
Jul 28 12:51:27.054684 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.054666    2562 manager.go:354] Recovery completed
Jul 28 12:51:27.059014 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.059000    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.060703 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.060682    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.060806 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.060720    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.060806 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.060734    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.061244 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061230    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 12:51:27.061244 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061241    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 12:51:27.061244 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061251    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.061403 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061256    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:51:27.061921 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061907    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.061978 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061933    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.061978 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.061944    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.062458 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.062378    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-196.ec2.internal.18c6752109f998c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-196.ec2.internal,UID:ip-10-0-132-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-196.ec2.internal,},FirstTimestamp:2026-07-28 12:51:27.060703425 +0000 UTC m=+0.377722416,LastTimestamp:2026-07-28 12:51:27.060703425 +0000 UTC m=+0.377722416,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-196.ec2.internal,}"
Jul 28 12:51:27.062837 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.062825    2562 policy_none.go:49] "None policy: Start"
Jul 28 12:51:27.062894 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.062840    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 12:51:27.062894 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.062851    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 12:51:27.063847 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.063837    2562 policy_none.go:47] "Start"
Jul 28 12:51:27.105756 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.105525    2562 manager.go:337] "Starting Device Plugin manager"
Jul 28 12:51:27.105887 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.105778    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 12:51:27.105887 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.105789    2562 server.go:87] "Starting device plugin registration server"
Jul 28 12:51:27.106050 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.106039    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 12:51:27.106094 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.106052    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 12:51:27.106199 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.106182    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 12:51:27.106284 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.106268    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 12:51:27.106284 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.106277    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 12:51:27.106856 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.106831    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 12:51:27.106940 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.106882    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.207058 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.206982    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.208149 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.207981    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.208149 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.208021    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.208149 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.208033    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.208149 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.208064    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.220727 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.220704    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.220840 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.220731    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-196.ec2.internal\": node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.238829 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.238803    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.243877 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.243855    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 12:51:27.245251 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.245230    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 12:51:27.245358 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.245266    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 12:51:27.245358 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.245287    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 12:51:27.245358 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.245320    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 12:51:27.250787 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.250762    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:51:27.339185 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.339153    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.346430 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.346394    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"]
Jul 28 12:51:27.346506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.346493    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.348059 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.348036    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.348166 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.348071    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.348166 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.348082    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.349566 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.349553    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.349763 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.349732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.349815 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.349777    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.350349 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350331    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.350462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350335    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.350462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350382    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.350462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350392    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.350462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350361    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.350462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.350441    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.351557 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.351543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.351641 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.351570    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:51:27.352279 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.352266    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:51:27.352367 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.352298    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:51:27.352367 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.352312    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:51:27.367341 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.367315    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-196.ec2.internal\" not found" node="ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.373242 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.373221    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-196.ec2.internal\" not found" node="ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.439799 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.439766    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.440939 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.440918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.441023 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.440952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.441023 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.440979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e60603cd4d0f2a54399ab7713287926-config\") pod \"kube-apiserver-proxy-ip-10-0-132-196.ec2.internal\" (UID: \"1e60603cd4d0f2a54399ab7713287926\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.539985 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.539907    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.542148 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.542198 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.542198 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e60603cd4d0f2a54399ab7713287926-config\") pod \"kube-apiserver-proxy-ip-10-0-132-196.ec2.internal\" (UID: \"1e60603cd4d0f2a54399ab7713287926\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.542258 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e60603cd4d0f2a54399ab7713287926-config\") pod \"kube-apiserver-proxy-ip-10-0-132-196.ec2.internal\" (UID: \"1e60603cd4d0f2a54399ab7713287926\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.542258 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.542320 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.542279    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a883dbf039b5028d3d5073d28b2091c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal\" (UID: \"2a883dbf039b5028d3d5073d28b2091c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.640400 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.640363    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.669705 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.669673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.676458 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.676434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:27.740718 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.740684    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.841811 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.841727    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.943228 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:27.943195    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:27.982449 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.982416    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 12:51:27.982967 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:27.982631    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 12:51:28.043952 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:28.043920    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-196.ec2.internal\" not found"
Jul 28 12:51:28.047385 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.047367    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:51:28.058543 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.058513    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 12:46:27 +0000 UTC" deadline="2028-05-10 23:00:33.12288887 +0000 UTC"
Jul 28 12:51:28.058543 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.058540    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15658h9m5.064351966s"
Jul 28 12:51:28.138897 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.138819    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:28.152404 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.152374    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:51:28.154609 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.154589    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal"
Jul 28 12:51:28.165750 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.165725    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:51:28.315859 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:28.315826    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e60603cd4d0f2a54399ab7713287926.slice/crio-632c675ec201c67a407d50f083b789ff52604afd7e9680dcce2d99f81c984aba WatchSource:0}: Error finding container 632c675ec201c67a407d50f083b789ff52604afd7e9680dcce2d99f81c984aba: Status 404 returned error can't find the container with id 632c675ec201c67a407d50f083b789ff52604afd7e9680dcce2d99f81c984aba
Jul 28 12:51:28.316320 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:28.316304    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a883dbf039b5028d3d5073d28b2091c.slice/crio-6f713e96eb583ffec5e80d9f770956b915c20fb0825af5a4ca3fcb748907508e WatchSource:0}: Error finding container 6f713e96eb583ffec5e80d9f770956b915c20fb0825af5a4ca3fcb748907508e: Status 404 returned error can't find the container with id 6f713e96eb583ffec5e80d9f770956b915c20fb0825af5a4ca3fcb748907508e
Jul 28 12:51:28.323708 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.323685    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:51:28.337414 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.337383    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:51:28.436279 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:28.436174    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:51:29.021811 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.021754    2562 apiserver.go:52] "Watching apiserver"
Jul 28 12:51:29.029502 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.029473    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 12:51:29.030941 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.030907    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-xbwq9","openshift-multus/multus-k5kzj","openshift-network-diagnostics/network-check-target-2bhz6","openshift-network-operator/iptables-alerter-lflws","kube-system/global-pull-secret-syncer-fv5kv","kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn","openshift-dns/node-resolver-jslj5","openshift-image-registry/node-ca-dkmbx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal","openshift-multus/network-metrics-daemon-2v6r2","openshift-ovn-kubernetes/ovnkube-node-m4mcc","kube-system/konnectivity-agent-mfdq5","openshift-cluster-node-tuning-operator/tuned-6jtb6"]
Jul 28 12:51:29.032696 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.032671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.033963 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.033937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.034957 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.034936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:29.035045 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.035011    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:29.036206 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.036182    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 12:51:29.036619 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.036597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.036892 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.036875    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.037435 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.037417    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lfl66\""
Jul 28 12:51:29.037645 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.037629    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 12:51:29.038220 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.038202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 12:51:29.038701 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.038684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b8c65\""
Jul 28 12:51:29.038826 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.038795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.039233 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.039214    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 12:51:29.043670 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.043424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.043670 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.043489    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:29.044967 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.044557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.046059 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.045936    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.046216 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.046197    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.046295 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.046245    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-48l75\""
Jul 28 12:51:29.046411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.046393    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 12:51:29.046556 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.046541    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.047173 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.047147    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.047366 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.047348    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.054988 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.054951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055110 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.055110 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:29.055110 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055182    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4deb847e-8bcb-47ba-869f-2f4bde506df0-agent-certs\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.055267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20fea077-7996-4cbb-bdea-19e471831c9b-host-slash\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.055267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-device-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4deb847e-8bcb-47ba-869f-2f4bde506df0-konnectivity-ca\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055321    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20fea077-7996-4cbb-bdea-19e471831c9b-iptables-alerter-script\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgclh\" (UniqueName: \"kubernetes.io/projected/20fea077-7996-4cbb-bdea-19e471831c9b-kube-api-access-dgclh\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-kubelet-config\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-dbus\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.055549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.055481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmk8t\" (UniqueName: \"kubernetes.io/projected/60a9cb29-259f-4da6-bc5b-48677491f6f4-kube-api-access-nmk8t\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.057100 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057071    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9m49c\""
Jul 28 12:51:29.057248 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057198    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.057578 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057554    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 12:51:29.057673 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057645    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kc55j\""
Jul 28 12:51:29.057905 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057882    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.058005 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.057936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.062840 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.062767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.062962 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.062909    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 12:51:29.063035 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.063016    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.063175 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.063157    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 12:51:29.064835 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.064419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.064835 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.064738    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.065492 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.065468    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.065805 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.065749    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:29.065978 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.065921    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 12:51:29.066051 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.065551    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2qj7r\""
Jul 28 12:51:29.066194 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.065823    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9vbtr\""
Jul 28 12:51:29.066194 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.066163    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 12:51:29.066563 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.066501    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:51:29.067325 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.067203    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.067325 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.067233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.068075 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.068054    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jpvcn\""
Jul 28 12:51:29.068540 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.068520    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.071194 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.071174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 12:51:29.071626 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.071607    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 12:51:29.071764 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.071742    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 12:51:29.071848 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.071757    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rcmhh\""
Jul 28 12:51:29.073018 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.072995    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 12:51:29.073257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.073239    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 12:51:29.076278 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.076256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 12:51:29.091198 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.090860    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjbzm"
Jul 28 12:51:29.099643 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.099619    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjbzm"
Jul 28 12:51:29.140034 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.140008    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 12:51:29.156465 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgclh\" (UniqueName: \"kubernetes.io/projected/20fea077-7996-4cbb-bdea-19e471831c9b-kube-api-access-dgclh\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.156647 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-dbus\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.156647 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.156647 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-tuned\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.156647 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85a6d0b-63f6-4e44-869f-7043450252d6-serviceca\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.156647 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-etc-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-host\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-os-release\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156794    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8mvf\" (UniqueName: \"kubernetes.io/projected/2fdfd184-4960-4e0e-975b-e70415a99242-kube-api-access-g8mvf\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-dbus\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85a6d0b-63f6-4e44-869f-7043450252d6-host\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156914    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-conf\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.156950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-lib-modules\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-conf-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.156986    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.156994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4deb847e-8bcb-47ba-869f-2f4bde506df0-agent-certs\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.157049    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:29.657030061 +0000 UTC m=+2.974049030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20fea077-7996-4cbb-bdea-19e471831c9b-host-slash\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-etc-kubernetes\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j54c\" (UniqueName: \"kubernetes.io/projected/65622881-6939-4444-8d8f-738998b063ef-kube-api-access-5j54c\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-kubelet\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-tmp\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-k8s-cni-cncf-io\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmk8t\" (UniqueName: \"kubernetes.io/projected/60a9cb29-259f-4da6-bc5b-48677491f6f4-kube-api-access-nmk8t\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cnibin\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.157509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20fea077-7996-4cbb-bdea-19e471831c9b-host-slash\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-log-socket\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157407    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-multus-certs\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d49j\" (UniqueName: \"kubernetes.io/projected/87a22f14-9114-4f72-a467-40611a128b83-kube-api-access-5d49j\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-os-release\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-systemd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85fr9\" (UniqueName: \"kubernetes.io/projected/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-kube-api-access-85fr9\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-kubelet\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-netns\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-systemd\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.158411 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-device-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-kubelet-config\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-slash\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-device-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.157973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-bin\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe4d3851-5860-4c58-9921-741f4a99e615-kubelet-config\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-socket-dir-parent\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-multus-daemon-config\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-netns\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-ovn\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-sys\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a22f14-9114-4f72-a467-40611a128b83-tmp-dir\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-config\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovn-node-metrics-cert\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.159383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-node-log\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-netd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-bin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x68l\" (UniqueName: \"kubernetes.io/projected/d85a6d0b-63f6-4e44-869f-7043450252d6-kube-api-access-9x68l\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-kubernetes\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-var-lib-kubelet\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-cnibin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysconfig\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158570    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfgj5\" (UniqueName: \"kubernetes.io/projected/96d11ec7-4ceb-426e-b8df-745b98e2abe0-kube-api-access-tfgj5\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-systemd-units\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-var-lib-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-env-overrides\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.160299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-script-lib\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fc4j\" (UniqueName: \"kubernetes.io/projected/b0701dd0-27ed-4253-b6b4-2e981a155ef6-kube-api-access-6fc4j\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-cni-binary-copy\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158880    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-multus\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-hostroot\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87a22f14-9114-4f72-a467-40611a128b83-hosts-file\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.158980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-modprobe-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159063    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60a9cb29-259f-4da6-bc5b-48677491f6f4-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-run\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-system-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4deb847e-8bcb-47ba-869f-2f4bde506df0-konnectivity-ca\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.159274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20fea077-7996-4cbb-bdea-19e471831c9b-iptables-alerter-script\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.160045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4deb847e-8bcb-47ba-869f-2f4bde506df0-konnectivity-ca\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.161001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.160070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20fea077-7996-4cbb-bdea-19e471831c9b-iptables-alerter-script\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.161696 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.161665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4deb847e-8bcb-47ba-869f-2f4bde506df0-agent-certs\") pod \"konnectivity-agent-mfdq5\" (UID: \"4deb847e-8bcb-47ba-869f-2f4bde506df0\") " pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.170848 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.170779    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:29.170848 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.170805    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:29.170848 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.170843    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:29.171094 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.170915    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:29.6708931 +0000 UTC m=+2.987912086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:29.171970 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.171946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgclh\" (UniqueName: \"kubernetes.io/projected/20fea077-7996-4cbb-bdea-19e471831c9b-kube-api-access-dgclh\") pod \"iptables-alerter-lflws\" (UID: \"20fea077-7996-4cbb-bdea-19e471831c9b\") " pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.172545 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.172517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmk8t\" (UniqueName: \"kubernetes.io/projected/60a9cb29-259f-4da6-bc5b-48677491f6f4-kube-api-access-nmk8t\") pod \"aws-ebs-csi-driver-node-jlkdn\" (UID: \"60a9cb29-259f-4da6-bc5b-48677491f6f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.251354 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.251289    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal" event={"ID":"1e60603cd4d0f2a54399ab7713287926","Type":"ContainerStarted","Data":"632c675ec201c67a407d50f083b789ff52604afd7e9680dcce2d99f81c984aba"}
Jul 28 12:51:29.252858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.252828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal" event={"ID":"2a883dbf039b5028d3d5073d28b2091c","Type":"ContainerStarted","Data":"6f713e96eb583ffec5e80d9f770956b915c20fb0825af5a4ca3fcb748907508e"}
Jul 28 12:51:29.260429 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-etc-kubernetes\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j54c\" (UniqueName: \"kubernetes.io/projected/65622881-6939-4444-8d8f-738998b063ef-kube-api-access-5j54c\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-kubelet\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-tmp\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.260565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-etc-kubernetes\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-kubelet\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260603    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-k8s-cni-cncf-io\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-k8s-cni-cncf-io\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cnibin\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-log-socket\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-multus-certs\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d49j\" (UniqueName: \"kubernetes.io/projected/87a22f14-9114-4f72-a467-40611a128b83-kube-api-access-5d49j\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-os-release\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-systemd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-multus-certs\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85fr9\" (UniqueName: \"kubernetes.io/projected/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-kube-api-access-85fr9\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-log-socket\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.260932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-kubelet\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-netns\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.260996    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-kubelet\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-systemd\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-slash\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-os-release\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-slash\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-bin\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-socket-dir-parent\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-systemd\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-multus-daemon-config\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-bin\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-socket-dir-parent\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-netns\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.261739 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-systemd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-ovn\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cnibin\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-sys\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-netns\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261281    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-run-ovn\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-run-netns\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a22f14-9114-4f72-a467-40611a128b83-tmp-dir\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-config\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261333    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-sys\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovn-node-metrics-cert\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-node-log\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-netd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261597    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a22f14-9114-4f72-a467-40611a128b83-tmp-dir\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-bin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x68l\" (UniqueName: \"kubernetes.io/projected/d85a6d0b-63f6-4e44-869f-7043450252d6-kube-api-access-9x68l\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.262667 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-host-cni-netd\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-kubernetes\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-var-lib-kubelet\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-cnibin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysconfig\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-multus-daemon-config\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261856    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfgj5\" (UniqueName: \"kubernetes.io/projected/96d11ec7-4ceb-426e-b8df-745b98e2abe0-kube-api-access-tfgj5\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-systemd-units\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-node-log\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-var-lib-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-cnibin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-env-overrides\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-script-lib\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-kubernetes\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fc4j\" (UniqueName: \"kubernetes.io/projected/b0701dd0-27ed-4253-b6b4-2e981a155ef6-kube-api-access-6fc4j\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-config\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.263686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.261923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-bin\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-var-lib-kubelet\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysconfig\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-cni-binary-copy\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-var-lib-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-multus\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-hostroot\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-systemd-units\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87a22f14-9114-4f72-a467-40611a128b83-hosts-file\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-modprobe-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-run\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-system-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-tuned\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85a6d0b-63f6-4e44-869f-7043450252d6-serviceca\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-etc-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.264549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-host\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-os-release\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8mvf\" (UniqueName: \"kubernetes.io/projected/2fdfd184-4960-4e0e-975b-e70415a99242-kube-api-access-g8mvf\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85a6d0b-63f6-4e44-869f-7043450252d6-host\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-conf\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovnkube-script-lib\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-lib-modules\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262652    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-conf-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.262846    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.262910    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:29.762891495 +0000 UTC m=+3.079910466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.262930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-env-overrides\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263089    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85a6d0b-63f6-4e44-869f-7043450252d6-host\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-multus-conf-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65622881-6939-4444-8d8f-738998b063ef-cni-binary-copy\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85a6d0b-63f6-4e44-869f-7043450252d6-serviceca\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.265159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-sysctl-conf\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d11ec7-4ceb-426e-b8df-745b98e2abe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-host\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-lib-modules\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-hostroot\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-tmp\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-system-cni-dir\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-host-var-lib-cni-multus\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-modprobe-d\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0701dd0-27ed-4253-b6b4-2e981a155ef6-run\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263453    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87a22f14-9114-4f72-a467-40611a128b83-hosts-file\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-etc-openvswitch\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263659    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d11ec7-4ceb-426e-b8df-745b98e2abe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.263358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65622881-6939-4444-8d8f-738998b063ef-os-release\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.264316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-ovn-node-metrics-cert\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.265734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.265556    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0701dd0-27ed-4253-b6b4-2e981a155ef6-etc-tuned\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.271523 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.271467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85fr9\" (UniqueName: \"kubernetes.io/projected/d0a1bb25-d2b4-45d4-a50e-f0ee622be190-kube-api-access-85fr9\") pod \"ovnkube-node-m4mcc\" (UID: \"d0a1bb25-d2b4-45d4-a50e-f0ee622be190\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.272363 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.272304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d49j\" (UniqueName: \"kubernetes.io/projected/87a22f14-9114-4f72-a467-40611a128b83-kube-api-access-5d49j\") pod \"node-resolver-jslj5\" (UID: \"87a22f14-9114-4f72-a467-40611a128b83\") " pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.273253 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.273221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfgj5\" (UniqueName: \"kubernetes.io/projected/96d11ec7-4ceb-426e-b8df-745b98e2abe0-kube-api-access-tfgj5\") pod \"multus-additional-cni-plugins-xbwq9\" (UID: \"96d11ec7-4ceb-426e-b8df-745b98e2abe0\") " pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.273612 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.273585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x68l\" (UniqueName: \"kubernetes.io/projected/d85a6d0b-63f6-4e44-869f-7043450252d6-kube-api-access-9x68l\") pod \"node-ca-dkmbx\" (UID: \"d85a6d0b-63f6-4e44-869f-7043450252d6\") " pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.274408 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.274085    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8mvf\" (UniqueName: \"kubernetes.io/projected/2fdfd184-4960-4e0e-975b-e70415a99242-kube-api-access-g8mvf\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.274668 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.274643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j54c\" (UniqueName: \"kubernetes.io/projected/65622881-6939-4444-8d8f-738998b063ef-kube-api-access-5j54c\") pod \"multus-k5kzj\" (UID: \"65622881-6939-4444-8d8f-738998b063ef\") " pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.275320 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.275296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fc4j\" (UniqueName: \"kubernetes.io/projected/b0701dd0-27ed-4253-b6b4-2e981a155ef6-kube-api-access-6fc4j\") pod \"tuned-6jtb6\" (UID: \"b0701dd0-27ed-4253-b6b4-2e981a155ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.363455 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.363412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn"
Jul 28 12:51:29.374684 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.374655    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:29.385649 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.385616    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jslj5"
Jul 28 12:51:29.391776 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.391728    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lflws"
Jul 28 12:51:29.401958 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.401928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6jtb6"
Jul 28 12:51:29.412277 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.411832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dkmbx"
Jul 28 12:51:29.417997 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.417974    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbwq9"
Jul 28 12:51:29.428177 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.428145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5kzj"
Jul 28 12:51:29.434435 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.434410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:29.537622 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.537533    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:51:29.666532 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.666491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:29.666721 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.666643    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:29.666816 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.666728    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:30.666707164 +0000 UTC m=+3.983726133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:29.767269 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.767220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:29.767474 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:29.767298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:29.767474 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767429    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:29.767474 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767435    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:29.767474 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767470    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:29.767705 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767485    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:29.767705 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767518    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:30.767483471 +0000 UTC m=+4.084502465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:29.767705 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:29.767544    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:30.767526581 +0000 UTC m=+4.084545554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:30.102106 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.102067    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:46:29 +0000 UTC" deadline="2028-03-03 16:32:16.3079954 +0000 UTC"
Jul 28 12:51:30.102106 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.102100    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14019h40m46.205898172s"
Jul 28 12:51:30.246531 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.246436    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:30.246531 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.246482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:30.246785 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.246592    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:30.246857 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.246785    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:30.351199 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.350962    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85a6d0b_63f6_4e44_869f_7043450252d6.slice/crio-f99d78b34a94caee1957b3c9668b37731a54c343c8ae7615d164d969b03b001d WatchSource:0}: Error finding container f99d78b34a94caee1957b3c9668b37731a54c343c8ae7615d164d969b03b001d: Status 404 returned error can't find the container with id f99d78b34a94caee1957b3c9668b37731a54c343c8ae7615d164d969b03b001d
Jul 28 12:51:30.379652 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.379617    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a9cb29_259f_4da6_bc5b_48677491f6f4.slice/crio-ed0956a3f51f83bd320e21ddc4ffa2146d5062dafd89ff781087df88032dce94 WatchSource:0}: Error finding container ed0956a3f51f83bd320e21ddc4ffa2146d5062dafd89ff781087df88032dce94: Status 404 returned error can't find the container with id ed0956a3f51f83bd320e21ddc4ffa2146d5062dafd89ff781087df88032dce94
Jul 28 12:51:30.381047 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.381023    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65622881_6939_4444_8d8f_738998b063ef.slice/crio-cdd8640f4d8d32e7a93a1baf0b932a76c23cd5260e46951343e9674c68e4332a WatchSource:0}: Error finding container cdd8640f4d8d32e7a93a1baf0b932a76c23cd5260e46951343e9674c68e4332a: Status 404 returned error can't find the container with id cdd8640f4d8d32e7a93a1baf0b932a76c23cd5260e46951343e9674c68e4332a
Jul 28 12:51:30.388348 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.388318    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a22f14_9114_4f72_a467_40611a128b83.slice/crio-8f157cf6f4c06f9fd3f85ee2ade05218e8b794c7204ae6fa84e070740998a4ae WatchSource:0}: Error finding container 8f157cf6f4c06f9fd3f85ee2ade05218e8b794c7204ae6fa84e070740998a4ae: Status 404 returned error can't find the container with id 8f157cf6f4c06f9fd3f85ee2ade05218e8b794c7204ae6fa84e070740998a4ae
Jul 28 12:51:30.389464 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.389439    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4deb847e_8bcb_47ba_869f_2f4bde506df0.slice/crio-14cec99e01f421c4e1640fd8fc46218b7a5b76f4171bbbbf2743179f782b66a3 WatchSource:0}: Error finding container 14cec99e01f421c4e1640fd8fc46218b7a5b76f4171bbbbf2743179f782b66a3: Status 404 returned error can't find the container with id 14cec99e01f421c4e1640fd8fc46218b7a5b76f4171bbbbf2743179f782b66a3
Jul 28 12:51:30.391075 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:51:30.390713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a1bb25_d2b4_45d4_a50e_f0ee622be190.slice/crio-3988af20a39c12de199cabafa88101903646122086a0fe4b57a2a0715377be96 WatchSource:0}: Error finding container 3988af20a39c12de199cabafa88101903646122086a0fe4b57a2a0715377be96: Status 404 returned error can't find the container with id 3988af20a39c12de199cabafa88101903646122086a0fe4b57a2a0715377be96
Jul 28 12:51:30.675077 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.674838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:30.675255 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.675008    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:30.675255 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.675214    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:32.675197111 +0000 UTC m=+5.992216084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:30.777084 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.777039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:30.777263 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:30.777105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:30.777263 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777208    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:30.777263 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777218    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:30.777263 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777242    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:30.777263 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777252    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:30.777429 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777269    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:32.777254562 +0000 UTC m=+6.094273531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:30.777429 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:30.777295    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:32.777282702 +0000 UTC m=+6.094301671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:31.102952 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.102870    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:46:29 +0000 UTC" deadline="2028-01-11 03:52:01.147219333 +0000 UTC"
Jul 28 12:51:31.102952 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.102939    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12759h0m30.044285277s"
Jul 28 12:51:31.248389 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.248348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:31.248592 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:31.248510    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:31.276582 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.276540    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal" event={"ID":"2a883dbf039b5028d3d5073d28b2091c","Type":"ContainerDied","Data":"82cfebae65b7cd77228659a9584ed34e125f06f9ed9d8083e906abd4a8527e48"}
Jul 28 12:51:31.277393 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.276505    2562 generic.go:356] "Generic (PLEG): container finished" podID="2a883dbf039b5028d3d5073d28b2091c" containerID="82cfebae65b7cd77228659a9584ed34e125f06f9ed9d8083e906abd4a8527e48" exitCode=0
Jul 28 12:51:31.303448 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.303411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lflws" event={"ID":"20fea077-7996-4cbb-bdea-19e471831c9b","Type":"ContainerStarted","Data":"87eee46407e9214a5fd75dbedc9cd4026b8aa5d43c935adf0cfae2f74883c557"}
Jul 28 12:51:31.312682 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.312637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerStarted","Data":"07346f8a81e266f62488739681d5031e7e495de321698d7dfbc81cb6c5d75717"}
Jul 28 12:51:31.328480 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.328371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfdq5" event={"ID":"4deb847e-8bcb-47ba-869f-2f4bde506df0","Type":"ContainerStarted","Data":"14cec99e01f421c4e1640fd8fc46218b7a5b76f4171bbbbf2743179f782b66a3"}
Jul 28 12:51:31.334527 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.334391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"3988af20a39c12de199cabafa88101903646122086a0fe4b57a2a0715377be96"}
Jul 28 12:51:31.352330 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.352291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal" event={"ID":"1e60603cd4d0f2a54399ab7713287926","Type":"ContainerStarted","Data":"f9fd1eabe79c76a0cc8eb2ec9f6c286d6adfd09cde6c7af6ddfad4d2c01bfd8e"}
Jul 28 12:51:31.361175 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.361076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6jtb6" event={"ID":"b0701dd0-27ed-4253-b6b4-2e981a155ef6","Type":"ContainerStarted","Data":"55f30f07e30a79bb901b0d9f4eea5b84da936e5ea2e3945ec5765c5a54affc69"}
Jul 28 12:51:31.378620 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.378578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jslj5" event={"ID":"87a22f14-9114-4f72-a467-40611a128b83","Type":"ContainerStarted","Data":"8f157cf6f4c06f9fd3f85ee2ade05218e8b794c7204ae6fa84e070740998a4ae"}
Jul 28 12:51:31.386419 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.386377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5kzj" event={"ID":"65622881-6939-4444-8d8f-738998b063ef","Type":"ContainerStarted","Data":"cdd8640f4d8d32e7a93a1baf0b932a76c23cd5260e46951343e9674c68e4332a"}
Jul 28 12:51:31.388785 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.388752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn" event={"ID":"60a9cb29-259f-4da6-bc5b-48677491f6f4","Type":"ContainerStarted","Data":"ed0956a3f51f83bd320e21ddc4ffa2146d5062dafd89ff781087df88032dce94"}
Jul 28 12:51:31.399470 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:31.399410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dkmbx" event={"ID":"d85a6d0b-63f6-4e44-869f-7043450252d6","Type":"ContainerStarted","Data":"f99d78b34a94caee1957b3c9668b37731a54c343c8ae7615d164d969b03b001d"}
Jul 28 12:51:32.246299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.246261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:32.246299 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.246261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:32.246903 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.246411    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:32.247028 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.246962    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:32.425056 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.424974    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal" event={"ID":"2a883dbf039b5028d3d5073d28b2091c","Type":"ContainerStarted","Data":"b649402e022f9b040f6e2abb17126481197453dc1b2c35bd4a3b4057b84c0e21"}
Jul 28 12:51:32.444204 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.444131    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-196.ec2.internal" podStartSLOduration=3.444109526 podStartE2EDuration="3.444109526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:32.444058519 +0000 UTC m=+5.761077513" watchObservedRunningTime="2026-07-28 12:51:32.444109526 +0000 UTC m=+5.761128521"
Jul 28 12:51:32.445092 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.445049    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-196.ec2.internal" podStartSLOduration=3.445035144 podStartE2EDuration="3.445035144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:31.376446796 +0000 UTC m=+4.693465791" watchObservedRunningTime="2026-07-28 12:51:32.445035144 +0000 UTC m=+5.762054140"
Jul 28 12:51:32.697524 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.697431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:32.697711 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.697650    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:32.697773 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.697725    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:36.697703803 +0000 UTC m=+10.014722773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:32.798203 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.798161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:32.798410 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:32.798247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:32.798474 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.798424    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:32.798533 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.798489    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:36.798469854 +0000 UTC m=+10.115488827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:32.798963 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.798941    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:32.799066 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.798974    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:32.799066 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.798990    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:32.799066 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:32.799040    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:36.799023256 +0000 UTC m=+10.116042229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:33.248789 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:33.248110    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:33.248789 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:33.248289    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:34.245987 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:34.245945    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:34.246214 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:34.246010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:34.246214 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:34.246127    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:34.246343 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:34.246271    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:35.249431 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:35.248878    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:35.249431 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:35.249036    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:36.245862 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:36.245826    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:36.246045 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.245967    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:36.246045 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:36.246041    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:36.246187 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.246117    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:36.737530 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:36.737423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:36.738012 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.737620    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:36.738012 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.737701    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:44.737676349 +0000 UTC m=+18.054695328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:36.838103 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:36.838052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:36.838126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838237    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838239    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838267    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838282    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838297    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:44.838277639 +0000 UTC m=+18.155296609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:36.838317 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:36.838324    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:51:44.838312505 +0000 UTC m=+18.155331474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:37.247963 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:37.247928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:37.248167 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:37.248026    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:38.246437 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:38.245925    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:38.246437 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:38.246057    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:38.246437 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:38.246089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:38.246437 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:38.246217    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:39.249762 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:39.248095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:39.249762 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:39.248246    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:40.246068 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:40.245501    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:40.246068 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:40.245619    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:40.246068 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:40.245957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:40.246068 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:40.246024    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:41.245909 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:41.245859    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:41.246342 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:41.246018    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:42.246226 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:42.246191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:42.246697 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:42.246190    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:42.246697 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:42.246322    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:42.246697 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:42.246419    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:43.245598 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:43.245558    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:43.245807 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:43.245715    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:44.245683 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:44.245644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:44.246192 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:44.245644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:44.246192 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.245802    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:44.246192 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.245849    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:44.804140 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:44.804102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:44.804319 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.804247    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:44.804319 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.804317    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:00.804299415 +0000 UTC m=+34.121318382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:51:44.904787 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:44.904731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:44.904962 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:44.904823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:44.904962 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904901    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:51:44.904962 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904913    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:44.904962 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904925    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:51:44.904962 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904935    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:44.905109 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904975    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:00.904960086 +0000 UTC m=+34.221979055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:51:44.905109 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:44.904988    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:00.904982586 +0000 UTC m=+34.222001554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:51:45.246346 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:45.246268    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:45.246797 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:45.246428    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:46.245808 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:46.245762    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:46.246007 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:46.245762    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:46.246007 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:46.245911    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:46.246007 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:46.245988    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:47.246750 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:47.246699    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:47.247226 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:47.246831    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:48.246016 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:48.245978    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:48.246172 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:48.245986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:48.246172 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:48.246095    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:48.246242 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:48.246207    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:49.246085 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:49.246052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:49.246665 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:49.246210    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:50.246314 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.246113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:50.247175 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.246181    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:50.247175 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:50.246408    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:50.247175 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:50.246481    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:50.468837 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.468803    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="81eaa0b03600d6a45b7e264305f82d500e756d027a9df3ba32e359d1e3247d64" exitCode=0
Jul 28 12:51:50.469000 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.468860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"81eaa0b03600d6a45b7e264305f82d500e756d027a9df3ba32e359d1e3247d64"}
Jul 28 12:51:50.470601 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.470463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfdq5" event={"ID":"4deb847e-8bcb-47ba-869f-2f4bde506df0","Type":"ContainerStarted","Data":"3aeb0e0c9a7632db2dd464d6afe89e6c34f57edfb58ed42370359b98a39b39e8"}
Jul 28 12:51:50.474551 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"0f770794f5a0b775258ad46927a47cab79eaa00521867e34904ebf5dee411549"}
Jul 28 12:51:50.476936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"71b9dfefa9c9fea2ac4bc0ad6d7bcfcc25be529a0f410329374ff249a94c1018"}
Jul 28 12:51:50.476936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474764    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"5d17525464ad8597dcb48307f7a3e4103647944fa8245372381765672b079a6e"}
Jul 28 12:51:50.476936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474858    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"a5fe0af46291b2e75eb9c37dc2286950ce89f534d8be6f1fb38393c9d5ed905a"}
Jul 28 12:51:50.476936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"83866b028fd9cbbb4cfdf03c1075098a55d5c74d4ef578c7e7387d0ee51911ad"}
Jul 28 12:51:50.476936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.474985    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"f6f89ff4684d99153f6679952bd12085e48914784e47d8dc1f3b498faa21e49a"}
Jul 28 12:51:50.478912 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.478880    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6jtb6" event={"ID":"b0701dd0-27ed-4253-b6b4-2e981a155ef6","Type":"ContainerStarted","Data":"f8457373be9acbf0c675fab533af55930e2f039be60cfc2d4636fdcca09a31f9"}
Jul 28 12:51:50.480371 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.480345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jslj5" event={"ID":"87a22f14-9114-4f72-a467-40611a128b83","Type":"ContainerStarted","Data":"2489018e898defad30b219353ed96a4b6ac71187522e4a5478e6fb0e01f0f61b"}
Jul 28 12:51:50.482029 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.482005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5kzj" event={"ID":"65622881-6939-4444-8d8f-738998b063ef","Type":"ContainerStarted","Data":"be7c8f0c521bf5d2adf4f060e87e0c5c5d53a7d16c8b3490f9d880dafcf8d078"}
Jul 28 12:51:50.483648 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.483612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn" event={"ID":"60a9cb29-259f-4da6-bc5b-48677491f6f4","Type":"ContainerStarted","Data":"dc9f58a96a9763b667141f19fffdc4ed097a8f1f8d707143618e73875f9fa9bf"}
Jul 28 12:51:50.485134 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.485111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dkmbx" event={"ID":"d85a6d0b-63f6-4e44-869f-7043450252d6","Type":"ContainerStarted","Data":"ebf4cb47c326d9f082640f52dbdf4b8f962e40ab053f53dccc5c8fcb7ec95fb7"}
Jul 28 12:51:50.522790 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.522683    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6jtb6" podStartSLOduration=3.422658153 podStartE2EDuration="22.522667711s" totalImagesPullingTime="19.100009558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:50.522114809 +0000 UTC m=+23.839133801" watchObservedRunningTime="2026-07-28 12:51:50.522667711 +0000 UTC m=+23.839686700"
Jul 28 12:51:50.571692 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.571650    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dkmbx" podStartSLOduration=3.450063244 podStartE2EDuration="22.571634028s" totalImagesPullingTime="19.121570784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:50.549641935 +0000 UTC m=+23.866660927" watchObservedRunningTime="2026-07-28 12:51:50.571634028 +0000 UTC m=+23.888653087"
Jul 28 12:51:50.571982 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.571956    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mfdq5" podStartSLOduration=7.550829855 podStartE2EDuration="22.571948957s" totalImagesPullingTime="15.021119102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:50.571492144 +0000 UTC m=+23.888511134" watchObservedRunningTime="2026-07-28 12:51:50.571948957 +0000 UTC m=+23.888967958"
Jul 28 12:51:50.597355 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.597308    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k5kzj" podStartSLOduration=3.394091564 podStartE2EDuration="22.597290006s" totalImagesPullingTime="19.203198442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:50.596926242 +0000 UTC m=+23.913945232" watchObservedRunningTime="2026-07-28 12:51:50.597290006 +0000 UTC m=+23.914308995"
Jul 28 12:51:50.617388 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.617341    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jslj5" podStartSLOduration=3.512514378 podStartE2EDuration="22.617325596s" totalImagesPullingTime="19.104811218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:50.616783487 +0000 UTC m=+23.933802476" watchObservedRunningTime="2026-07-28 12:51:50.617325596 +0000 UTC m=+23.934344585"
Jul 28 12:51:50.665840 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:50.665818    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 12:51:51.125188 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.124997    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T12:51:50.66583602Z","UUID":"5da4a4bf-3cfc-41bb-b3b8-ce3eeb9dfe97","Handler":null,"Name":"","Endpoint":""}
Jul 28 12:51:51.126791 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.126765    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 12:51:51.126939 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.126814    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 12:51:51.246191 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.246157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:51.246349 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:51.246309    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:51.490178 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.489556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn" event={"ID":"60a9cb29-259f-4da6-bc5b-48677491f6f4","Type":"ContainerStarted","Data":"6cc3b90ec59dbbe401233550486be157a97ebccd2ded77d43a450c6d756a5a6d"}
Jul 28 12:51:51.491812 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.491454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lflws" event={"ID":"20fea077-7996-4cbb-bdea-19e471831c9b","Type":"ContainerStarted","Data":"8381adb4a103294df5f06e441c851e7174fc9ae4719062959ed8325d27865082"}
Jul 28 12:51:51.992997 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.992962    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:51.993558 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:51.993539    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:52.014305 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:52.014196    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lflws" podStartSLOduration=4.919849564 podStartE2EDuration="24.014180983s" totalImagesPullingTime="19.094331419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:51.518346606 +0000 UTC m=+24.835365597" watchObservedRunningTime="2026-07-28 12:51:52.014180983 +0000 UTC m=+25.331200030"
Jul 28 12:51:52.245690 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:52.245651    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:52.245888 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:52.245695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:52.245888 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:52.245805    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:52.246218 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:52.246185    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:52.499753 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:52.499634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"75c0d5ee5d1c61d83c27c5907712ab0787bed3cf64e3a8e212390a4e1e721760"}
Jul 28 12:51:52.502520 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:52.502485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn" event={"ID":"60a9cb29-259f-4da6-bc5b-48677491f6f4","Type":"ContainerStarted","Data":"ca607706a542fcafc100993c6793ad861e2c4694a9bfb227bef3b6ab653ddd26"}
Jul 28 12:51:53.246413 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:53.246379    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:53.246590 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:53.246510    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:53.504949 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:53.504869    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 12:51:53.506985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:53.506958    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jslj5_87a22f14-9114-4f72-a467-40611a128b83/dns-node-resolver/0.log"
Jul 28 12:51:54.246248 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:54.246210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:54.246444 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:54.246210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:54.246444 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:54.246355    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:54.246545 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:54.246450    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:54.462816 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:54.462578    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dkmbx_d85a6d0b-63f6-4e44-869f-7043450252d6/node-ca/0.log"
Jul 28 12:51:55.246655 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.246564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:55.247376 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:55.246686    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:55.510037 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.509948    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="2312a9def925a2d34df6f7f30b716512a3893de328a32e7c2992a9f53845ba67" exitCode=0
Jul 28 12:51:55.510193 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.510035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"2312a9def925a2d34df6f7f30b716512a3893de328a32e7c2992a9f53845ba67"}
Jul 28 12:51:55.514092 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.513890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" event={"ID":"d0a1bb25-d2b4-45d4-a50e-f0ee622be190","Type":"ContainerStarted","Data":"57e93836a30ac64f5ef02b939eb1d6361f83d6ab5c5ec305f0c0e4b2941593ed"}
Jul 28 12:51:55.514229 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.514171    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:55.530223 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.530199    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:55.546379 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.546318    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkdn" podStartSLOduration=6.550728483 podStartE2EDuration="27.546302592s" totalImagesPullingTime="20.995574109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:52.521412645 +0000 UTC m=+25.838431634" watchObservedRunningTime="2026-07-28 12:51:55.546302592 +0000 UTC m=+28.863321574"
Jul 28 12:51:55.598228 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.598179    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc" podStartSLOduration=8.44087654 podStartE2EDuration="27.598163617s" totalImagesPullingTime="19.157287077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:51:55.596375169 +0000 UTC m=+28.913394158" watchObservedRunningTime="2026-07-28 12:51:55.598163617 +0000 UTC m=+28.915182612"
Jul 28 12:51:55.612610 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:55.612578    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:56.245651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.245612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:56.245908 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.245660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:56.245908 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:56.245758    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:56.245908 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:56.245821    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:56.516991 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.516913    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:56.533437 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.533405    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:51:56.857041 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.857013    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:56.857179 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.857144    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 12:51:56.857682 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:56.857657    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mfdq5"
Jul 28 12:51:57.246378 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:57.246294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:57.246526 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:57.246392    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:57.521086 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:57.520998    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="5fe33e9adf8295a653d4de7b39c4c580ea97fee4f146f44360f9bab0acb7564d" exitCode=0
Jul 28 12:51:57.521534 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:57.521087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"5fe33e9adf8295a653d4de7b39c4c580ea97fee4f146f44360f9bab0acb7564d"}
Jul 28 12:51:58.246223 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:58.246192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:51:58.246392 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:58.246192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:51:58.246392 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:58.246312    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:51:58.246392 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:58.246374    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:51:59.246296 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:59.246259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:51:59.246680 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:51:59.246376    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:51:59.528015 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:59.527922    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="7f908e91f7cdd2fb778bbd5c8f343a320de26940885ef998dbff1ab07b2576c0" exitCode=0
Jul 28 12:51:59.528015 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:51:59.527998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"7f908e91f7cdd2fb778bbd5c8f343a320de26940885ef998dbff1ab07b2576c0"}
Jul 28 12:52:00.246048 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:00.246014    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:00.246048 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:00.246054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:00.246302 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.246143    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:00.246302 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.246263    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:00.830096 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:00.830058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:00.830292 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.830195    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:52:00.830292 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.830265    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret podName:fe4d3851-5860-4c58-9921-741f4a99e615 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:32.830250991 +0000 UTC m=+66.147269959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret") pod "global-pull-secret-syncer-fv5kv" (UID: "fe4d3851-5860-4c58-9921-741f4a99e615") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:52:00.931201 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:00.931163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:00.931386 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:00.931234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:00.931386 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931333    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:52:00.931386 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931352    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:52:00.931386 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931369    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:52:00.931386 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931379    2562 projected.go:196] Error preparing data for projected volume kube-api-access-x6hlv for pod openshift-network-diagnostics/network-check-target-2bhz6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:52:00.931595 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931399    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs podName:2fdfd184-4960-4e0e-975b-e70415a99242 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:32.93138403 +0000 UTC m=+66.248403002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs") pod "network-metrics-daemon-2v6r2" (UID: "2fdfd184-4960-4e0e-975b-e70415a99242") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:52:00.931595 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:00.931427    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv podName:021805c8-b3fb-4e66-ba52-361313f279d5 nodeName:}" failed. No retries permitted until 2026-07-28 12:52:32.931410043 +0000 UTC m=+66.248429011 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6hlv" (UniqueName: "kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv") pod "network-check-target-2bhz6" (UID: "021805c8-b3fb-4e66-ba52-361313f279d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:52:01.246449 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:01.246351    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:01.246916 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:01.246486    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:02.245989 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:02.245497    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:02.245989 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:02.245551    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:02.245989 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:02.245635    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:02.245989 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:02.245764    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:03.246488 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:03.246454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:03.246988 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:03.246597    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:04.246187 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:04.246105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:04.246187 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:04.246151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:04.246375 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:04.246219    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:04.246375 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:04.246338    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:04.543315 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:04.543224    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="5922d2c8a2e06fede276db473f281726d32d7566be8450c6d897940e79e5779b" exitCode=0
Jul 28 12:52:04.543315 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:04.543289    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"5922d2c8a2e06fede276db473f281726d32d7566be8450c6d897940e79e5779b"}
Jul 28 12:52:05.246034 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:05.245995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:05.246283 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:05.246119    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:05.548655 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:05.548564    2562 generic.go:356] "Generic (PLEG): container finished" podID="96d11ec7-4ceb-426e-b8df-745b98e2abe0" containerID="e60369a300ed9dc9610f574f3d860758bedac54232ac254b9bf05384ff6ba798" exitCode=0
Jul 28 12:52:05.548655 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:05.548629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerDied","Data":"e60369a300ed9dc9610f574f3d860758bedac54232ac254b9bf05384ff6ba798"}
Jul 28 12:52:06.245674 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:06.245636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:06.245674 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:06.245676    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:06.245898 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:06.245778    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:06.245898 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:06.245861    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:06.554078 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:06.553995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" event={"ID":"96d11ec7-4ceb-426e-b8df-745b98e2abe0","Type":"ContainerStarted","Data":"6ab4ae97921ce8e8bcd41ab82c06b9624097f3f93ae865ed7dc96ecc0e3a247d"}
Jul 28 12:52:06.584139 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:06.584078    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xbwq9" podStartSLOduration=8.503867148 podStartE2EDuration="38.584062195s" totalImagesPullingTime="30.080195047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:06.583685114 +0000 UTC m=+39.900704103" watchObservedRunningTime="2026-07-28 12:52:06.584062195 +0000 UTC m=+39.901081185"
Jul 28 12:52:07.247032 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:07.246999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:07.247206 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:07.247091    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:08.246398 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:08.246359    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:08.246891 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:08.246361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:08.246891 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:08.246465    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:08.246891 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:08.246566    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:09.246529 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:09.246485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:09.246958 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:09.246641    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:10.246267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:10.246233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:10.246267 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:10.246261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:10.246482 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:10.246348    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:10.246482 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:10.246425    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:11.245762 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:11.245710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:11.246162 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:11.245843    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:12.246211 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:12.246173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:12.246619 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:12.246173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:12.246619 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:12.246287    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:12.246619 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:12.246357    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:13.246155 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:13.246122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:13.246329 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:13.246235    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:14.245974 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:14.245915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:14.246185 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:14.245926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:14.246185 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:14.246040    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:14.246185 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:14.246137    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:15.246433 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:15.246385    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:15.246849 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:15.246511    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:16.245794 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.245759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:16.245794 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.245782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:16.246014 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:16.245882    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:16.246014 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:16.245994    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:16.996177 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.995938    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bhz6"]
Jul 28 12:52:16.996781 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.996290    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:16.996781 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:16.996397    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:16.999734 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.999655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fv5kv"]
Jul 28 12:52:16.999891 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:16.999799    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:17.001050 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:16.999914    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:17.001050 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:17.000447    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2v6r2"]
Jul 28 12:52:17.001050 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:17.000539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:17.001050 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:17.000634    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:18.246431 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:18.246389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:18.247028 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:18.246392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:18.247028 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:18.246517    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:18.247028 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:18.246603    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:19.246508 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:19.246477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:19.247086 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:19.246589    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:20.245500 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:20.245461    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:20.245683 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:20.245576    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v6r2" podUID="2fdfd184-4960-4e0e-975b-e70415a99242"
Jul 28 12:52:20.245683 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:20.245605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:20.245683 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:20.245654    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bhz6" podUID="021805c8-b3fb-4e66-ba52-361313f279d5"
Jul 28 12:52:21.246107 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.246070    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:21.246671 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:21.246182    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fv5kv" podUID="fe4d3851-5860-4c58-9921-741f4a99e615"
Jul 28 12:52:21.522266 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.522236    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-196.ec2.internal" event="NodeReady"
Jul 28 12:52:21.522435 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.522384    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 12:52:21.597422 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.597387    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5pwgs"]
Jul 28 12:52:21.605333 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.605307    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mplhd"]
Jul 28 12:52:21.605473 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.605428    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.609274 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.608904    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qldgx\""
Jul 28 12:52:21.609974 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.609950    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 12:52:21.610089 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.609950    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 12:52:21.610523 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.610503    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 12:52:21.613706 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.613684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.615713 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.615692    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pwgs"]
Jul 28 12:52:21.616085 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.616070    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 12:52:21.616287 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.616268    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 12:52:21.616422 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.616405    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 12:52:21.616790 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.616776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wb5jb\""
Jul 28 12:52:21.617155 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.617143    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 12:52:21.623383 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.623363    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mplhd"]
Jul 28 12:52:21.674752 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.674940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0560dc-5cae-4942-90cd-d11f2e0de254-cert\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.674940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-crio-socket\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.674940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.674940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2hq\" (UniqueName: \"kubernetes.io/projected/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-api-access-nm2hq\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.674940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-data-volume\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.675110 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.674974    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb62z\" (UniqueName: \"kubernetes.io/projected/4b0560dc-5cae-4942-90cd-d11f2e0de254-kube-api-access-gb62z\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.699716 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.699686    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5g2pq"]
Jul 28 12:52:21.713427 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.713397    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.714205 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.714179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g2pq"]
Jul 28 12:52:21.716636 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.716617    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wxrjt\""
Jul 28 12:52:21.716909 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.716888    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 12:52:21.717000 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.716967    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 12:52:21.775658 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0560dc-5cae-4942-90cd-d11f2e0de254-cert\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.775658 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-metrics-tls\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-crio-socket\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-config-volume\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775727    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smt92\" (UniqueName: \"kubernetes.io/projected/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-kube-api-access-smt92\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775836    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2hq\" (UniqueName: \"kubernetes.io/projected/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-api-access-nm2hq\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.775917 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-data-volume\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.776159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-crio-socket\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.776159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.775932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb62z\" (UniqueName: \"kubernetes.io/projected/4b0560dc-5cae-4942-90cd-d11f2e0de254-kube-api-access-gb62z\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.776159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.776022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.776159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.776049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-tmp-dir\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.776344 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.776294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-data-volume\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.776591 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.776574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.779993 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.779972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0560dc-5cae-4942-90cd-d11f2e0de254-cert\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.790672 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.790634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb62z\" (UniqueName: \"kubernetes.io/projected/4b0560dc-5cae-4942-90cd-d11f2e0de254-kube-api-access-gb62z\") pod \"ingress-canary-5pwgs\" (UID: \"4b0560dc-5cae-4942-90cd-d11f2e0de254\") " pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.790843 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.790772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2hq\" (UniqueName: \"kubernetes.io/projected/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-kube-api-access-nm2hq\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.791220 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.791202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mplhd\" (UID: \"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e\") " pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:21.876995 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.876951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-tmp-dir\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.876995 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.876997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-metrics-tls\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.877210 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.877041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-config-volume\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.877210 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.877070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt92\" (UniqueName: \"kubernetes.io/projected/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-kube-api-access-smt92\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.877462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.877438    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-tmp-dir\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.877788 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.877758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-config-volume\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.879706 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.879677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-metrics-tls\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.887665 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.887640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smt92\" (UniqueName: \"kubernetes.io/projected/f68cd3a9-d5e2-46a2-95e1-fa630b6ab563-kube-api-access-smt92\") pod \"dns-default-5g2pq\" (UID: \"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563\") " pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:21.917825 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.917787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pwgs"
Jul 28 12:52:21.924954 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:21.924924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mplhd"
Jul 28 12:52:22.005833 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.005326    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"]
Jul 28 12:52:22.018936 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.018647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:22.022546 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.022346    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:22.023504 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.023270    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tmrpk\""
Jul 28 12:52:22.023954 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.023778    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 12:52:22.041467 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.041399    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"]
Jul 28 12:52:22.078578 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.078394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2lzkk\" (UID: \"25b7d641-ff4a-439e-926c-83c3a75a6e4c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:22.131944 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.131797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pwgs"]
Jul 28 12:52:22.135760 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.135718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mplhd"]
Jul 28 12:52:22.136708 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:22.136687    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0560dc_5cae_4942_90cd_d11f2e0de254.slice/crio-273fdced8d33ccf6ea007f8487d5debd987483002a8e02130b18d37c4402c0f5 WatchSource:0}: Error finding container 273fdced8d33ccf6ea007f8487d5debd987483002a8e02130b18d37c4402c0f5: Status 404 returned error can't find the container with id 273fdced8d33ccf6ea007f8487d5debd987483002a8e02130b18d37c4402c0f5
Jul 28 12:52:22.138989 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:22.138969    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b5b4f4_aea6_40b6_b8ce_8d99e755bc0e.slice/crio-b81a1b60cc28422f3e019f99403947a056c48d0fbe24b597b35ec13f401ad52b WatchSource:0}: Error finding container b81a1b60cc28422f3e019f99403947a056c48d0fbe24b597b35ec13f401ad52b: Status 404 returned error can't find the container with id b81a1b60cc28422f3e019f99403947a056c48d0fbe24b597b35ec13f401ad52b
Jul 28 12:52:22.179727 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.179687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2lzkk\" (UID: \"25b7d641-ff4a-439e-926c-83c3a75a6e4c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:22.179889 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:22.179838    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 12:52:22.179944 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:22.179913    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates podName:25b7d641-ff4a-439e-926c-83c3a75a6e4c nodeName:}" failed. No retries permitted until 2026-07-28 12:52:22.679893773 +0000 UTC m=+55.996912741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-2lzkk" (UID: "25b7d641-ff4a-439e-926c-83c3a75a6e4c") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 12:52:22.182956 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.182932    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g2pq"]
Jul 28 12:52:22.187008 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:22.186981    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68cd3a9_d5e2_46a2_95e1_fa630b6ab563.slice/crio-0a0788a3227f72664cc5ccf9eb6409bf065ca19e1aaec182b8ce1482121006a3 WatchSource:0}: Error finding container 0a0788a3227f72664cc5ccf9eb6409bf065ca19e1aaec182b8ce1482121006a3: Status 404 returned error can't find the container with id 0a0788a3227f72664cc5ccf9eb6409bf065ca19e1aaec182b8ce1482121006a3
Jul 28 12:52:22.246191 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.246159    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:22.246191 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.246195    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:22.250576 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.250554    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:52:22.250723 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.250599    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lcpbt\""
Jul 28 12:52:22.250723 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.250636    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:52:22.250723 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.250647    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:52:22.250723 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.250662    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n8x7n\""
Jul 28 12:52:22.594211 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.594173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mplhd" event={"ID":"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e","Type":"ContainerStarted","Data":"b81a1b60cc28422f3e019f99403947a056c48d0fbe24b597b35ec13f401ad52b"}
Jul 28 12:52:22.595260 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.595228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pwgs" event={"ID":"4b0560dc-5cae-4942-90cd-d11f2e0de254","Type":"ContainerStarted","Data":"273fdced8d33ccf6ea007f8487d5debd987483002a8e02130b18d37c4402c0f5"}
Jul 28 12:52:22.598140 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.598116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g2pq" event={"ID":"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563","Type":"ContainerStarted","Data":"0a0788a3227f72664cc5ccf9eb6409bf065ca19e1aaec182b8ce1482121006a3"}
Jul 28 12:52:22.684456 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.684420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2lzkk\" (UID: \"25b7d641-ff4a-439e-926c-83c3a75a6e4c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:22.687958 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.687933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b7d641-ff4a-439e-926c-83c3a75a6e4c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2lzkk\" (UID: \"25b7d641-ff4a-439e-926c-83c3a75a6e4c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:22.940288 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:22.940208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:23.119010 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:23.118944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"]
Jul 28 12:52:23.126921 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:23.126890    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b7d641_ff4a_439e_926c_83c3a75a6e4c.slice/crio-78eaad5418a0bb558aedbc1849139d70153b2a23247867a173c319e81b58fdb7 WatchSource:0}: Error finding container 78eaad5418a0bb558aedbc1849139d70153b2a23247867a173c319e81b58fdb7: Status 404 returned error can't find the container with id 78eaad5418a0bb558aedbc1849139d70153b2a23247867a173c319e81b58fdb7
Jul 28 12:52:23.249289 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:23.248677    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:23.251872 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:23.251846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:52:23.602620 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:23.602490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk" event={"ID":"25b7d641-ff4a-439e-926c-83c3a75a6e4c","Type":"ContainerStarted","Data":"78eaad5418a0bb558aedbc1849139d70153b2a23247867a173c319e81b58fdb7"}
Jul 28 12:52:28.537116 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.537087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4mcc"
Jul 28 12:52:28.617793 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.617690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mplhd" event={"ID":"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e","Type":"ContainerStarted","Data":"bd12a619621b3c660c253fa122da66efdf13c97a47f924ca3d9aaef47fdef278"}
Jul 28 12:52:28.619909 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.619870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pwgs" event={"ID":"4b0560dc-5cae-4942-90cd-d11f2e0de254","Type":"ContainerStarted","Data":"dfefff1e19c65b2bea597fa632ace086edd9088ef0824027d6c9d72ca47bd92d"}
Jul 28 12:52:28.622516 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.622483    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g2pq" event={"ID":"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563","Type":"ContainerStarted","Data":"2238b78c5ff27febd6791908babd65883db6f99f9ba5941e6863a5d07260b842"}
Jul 28 12:52:28.622648 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.622548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:28.622648 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.622572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g2pq" event={"ID":"f68cd3a9-d5e2-46a2-95e1-fa630b6ab563","Type":"ContainerStarted","Data":"643bace939b43fad00757ff0652aa953b42e9ccfb916c866a805776b760a7f6b"}
Jul 28 12:52:28.624044 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.624021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk" event={"ID":"25b7d641-ff4a-439e-926c-83c3a75a6e4c","Type":"ContainerStarted","Data":"152144c7f532d666ee49c95137cbb547ba4c250d9e4ba2fb4bab62bfd48a66d1"}
Jul 28 12:52:28.624256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.624237    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:28.629021 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.629004    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk"
Jul 28 12:52:28.641275 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.641225    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5pwgs" podStartSLOduration=1.428270494 podStartE2EDuration="7.641211567s" totalImagesPullingTime="6.212941073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:28.641115871 +0000 UTC m=+61.958134886" watchObservedRunningTime="2026-07-28 12:52:28.641211567 +0000 UTC m=+61.958230538"
Jul 28 12:52:28.670078 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.670035    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5g2pq" podStartSLOduration=1.50822813 podStartE2EDuration="7.670018504s" totalImagesPullingTime="6.161790374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:28.669137213 +0000 UTC m=+61.986156205" watchObservedRunningTime="2026-07-28 12:52:28.670018504 +0000 UTC m=+61.987037493"
Jul 28 12:52:28.707311 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:28.707267    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2lzkk" podStartSLOduration=2.490521416 podStartE2EDuration="7.707253655s" totalImagesPullingTime="5.216732239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:28.698439572 +0000 UTC m=+62.015458562" watchObservedRunningTime="2026-07-28 12:52:28.707253655 +0000 UTC m=+62.024272623"
Jul 28 12:52:29.094724 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.094689    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"]
Jul 28 12:52:29.096955 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.096932    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.100499 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.100478    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 12:52:29.101805 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.100991    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 12:52:29.101805 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.101168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 12:52:29.101805 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.101452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 12:52:29.101805 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.101617    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 12:52:29.102055 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.101961    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xv7fq\""
Jul 28 12:52:29.115452 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.115426    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"]
Jul 28 12:52:29.139143 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.138867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvzw\" (UniqueName: \"kubernetes.io/projected/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-kube-api-access-cfvzw\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.139143 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.138954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.139143 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.139001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.139143 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.139033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.239780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.239723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.239967 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.239814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.239967 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.239851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.239967 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.239901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvzw\" (UniqueName: \"kubernetes.io/projected/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-kube-api-access-cfvzw\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.241297 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.241266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.243212 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.243184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.243322 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.243254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.252490 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.252460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvzw\" (UniqueName: \"kubernetes.io/projected/64b5f305-e14b-4ac9-baff-ce97e9b5ba56-kube-api-access-cfvzw\") pod \"prometheus-operator-84d69f794d-kqs6f\" (UID: \"64b5f305-e14b-4ac9-baff-ce97e9b5ba56\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.409357 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.409280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"
Jul 28 12:52:29.559303 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.559265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-kqs6f"]
Jul 28 12:52:29.562427 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:29.562401    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b5f305_e14b_4ac9_baff_ce97e9b5ba56.slice/crio-e2bbe050e58e59ffead3806e038a3bda3e5bf6260aa0fbee6bdb2b7b36578e18 WatchSource:0}: Error finding container e2bbe050e58e59ffead3806e038a3bda3e5bf6260aa0fbee6bdb2b7b36578e18: Status 404 returned error can't find the container with id e2bbe050e58e59ffead3806e038a3bda3e5bf6260aa0fbee6bdb2b7b36578e18
Jul 28 12:52:29.627894 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:29.627852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f" event={"ID":"64b5f305-e14b-4ac9-baff-ce97e9b5ba56","Type":"ContainerStarted","Data":"e2bbe050e58e59ffead3806e038a3bda3e5bf6260aa0fbee6bdb2b7b36578e18"}
Jul 28 12:52:32.639278 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.639240    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mplhd" event={"ID":"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e","Type":"ContainerStarted","Data":"4f3b7a40731835077a5fb0788e8e12913b3e20f9305c8b9a63a4aefca7fe2314"}
Jul 28 12:52:32.640954 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.640929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f" event={"ID":"64b5f305-e14b-4ac9-baff-ce97e9b5ba56","Type":"ContainerStarted","Data":"42f4379b050b9da074c1056606bb5faea2a5e991eca67d1fb602e6d3bd74140b"}
Jul 28 12:52:32.641061 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.640978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f" event={"ID":"64b5f305-e14b-4ac9-baff-ce97e9b5ba56","Type":"ContainerStarted","Data":"d7247491e1c6625bd324450126bf3d6ed5f6461974b258e18214cb2745a2e1b0"}
Jul 28 12:52:32.662811 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.662759    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-kqs6f" podStartSLOduration=1.07814197 podStartE2EDuration="3.662725012s" totalImagesPullingTime="2.584583042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:32.661700322 +0000 UTC m=+65.978719312" watchObservedRunningTime="2026-07-28 12:52:32.662725012 +0000 UTC m=+65.979743995"
Jul 28 12:52:32.870190 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.870147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:32.872979 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.872945    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:52:32.883811 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.883788    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe4d3851-5860-4c58-9921-741f4a99e615-original-pull-secret\") pod \"global-pull-secret-syncer-fv5kv\" (UID: \"fe4d3851-5860-4c58-9921-741f4a99e615\") " pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:32.971109 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.971030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:32.971109 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.971092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:32.973991 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.973963    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:52:32.974134 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.974098    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:52:32.984152 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.984124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fdfd184-4960-4e0e-975b-e70415a99242-metrics-certs\") pod \"network-metrics-daemon-2v6r2\" (UID: \"2fdfd184-4960-4e0e-975b-e70415a99242\") " pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:32.984283 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.984195    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:52:32.994246 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:32.994218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hlv\" (UniqueName: \"kubernetes.io/projected/021805c8-b3fb-4e66-ba52-361313f279d5-kube-api-access-x6hlv\") pod \"network-check-target-2bhz6\" (UID: \"021805c8-b3fb-4e66-ba52-361313f279d5\") " pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:33.059076 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.059048    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lcpbt\""
Jul 28 12:52:33.064089 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.064052    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n8x7n\""
Jul 28 12:52:33.067341 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.067318    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:33.071365 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.071337    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v6r2"
Jul 28 12:52:33.162545 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.162452    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fv5kv"
Jul 28 12:52:33.225838 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.225768    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bhz6"]
Jul 28 12:52:33.233600 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:33.233565    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021805c8_b3fb_4e66_ba52_361313f279d5.slice/crio-054f67f66e49026fccb7553b27e505700b2fc982be1df6d2275b01efe9b16f9c WatchSource:0}: Error finding container 054f67f66e49026fccb7553b27e505700b2fc982be1df6d2275b01efe9b16f9c: Status 404 returned error can't find the container with id 054f67f66e49026fccb7553b27e505700b2fc982be1df6d2275b01efe9b16f9c
Jul 28 12:52:33.268043 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.268016    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2v6r2"]
Jul 28 12:52:33.273169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:33.272829    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdfd184_4960_4e0e_975b_e70415a99242.slice/crio-711236cba09c982a8e84dbfe4d182011d18829b371a542fd07482a79419b305b WatchSource:0}: Error finding container 711236cba09c982a8e84dbfe4d182011d18829b371a542fd07482a79419b305b: Status 404 returned error can't find the container with id 711236cba09c982a8e84dbfe4d182011d18829b371a542fd07482a79419b305b
Jul 28 12:52:33.322961 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.322931    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fv5kv"]
Jul 28 12:52:33.326919 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:33.326880    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d3851_5860_4c58_9921_741f4a99e615.slice/crio-82b19365b6a303bf589049ed7070f4ecf3e772db5742c52f3dae4e629d6a37de WatchSource:0}: Error finding container 82b19365b6a303bf589049ed7070f4ecf3e772db5742c52f3dae4e629d6a37de: Status 404 returned error can't find the container with id 82b19365b6a303bf589049ed7070f4ecf3e772db5742c52f3dae4e629d6a37de
Jul 28 12:52:33.647015 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.646972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bhz6" event={"ID":"021805c8-b3fb-4e66-ba52-361313f279d5","Type":"ContainerStarted","Data":"054f67f66e49026fccb7553b27e505700b2fc982be1df6d2275b01efe9b16f9c"}
Jul 28 12:52:33.648602 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.648563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fv5kv" event={"ID":"fe4d3851-5860-4c58-9921-741f4a99e615","Type":"ContainerStarted","Data":"82b19365b6a303bf589049ed7070f4ecf3e772db5742c52f3dae4e629d6a37de"}
Jul 28 12:52:33.651017 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:33.650970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v6r2" event={"ID":"2fdfd184-4960-4e0e-975b-e70415a99242","Type":"ContainerStarted","Data":"711236cba09c982a8e84dbfe4d182011d18829b371a542fd07482a79419b305b"}
Jul 28 12:52:34.499143 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.499104    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"]
Jul 28 12:52:34.503206 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.503180    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.507063 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.506480    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 12:52:34.507063 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.506661    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z2gjp\""
Jul 28 12:52:34.507063 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.506784    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:52:34.521440 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.521411    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"]
Jul 28 12:52:34.583791 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.583732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.583960 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.583800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a3186d-d911-4511-b390-bf1e454dc0ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.583960 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.583835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.583960 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.583862    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6bp\" (UniqueName: \"kubernetes.io/projected/b6a3186d-d911-4511-b390-bf1e454dc0ba-kube-api-access-cl6bp\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.613780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.612917    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"]
Jul 28 12:52:34.619729 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.619144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.619729 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.619197    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7dgsl"]
Jul 28 12:52:34.622438 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.621950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.623194 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.623175    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nhvdd\""
Jul 28 12:52:34.623910 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.623892    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 12:52:34.624111 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.624093    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 12:52:34.624185 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.624116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:52:34.624369 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.624353    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 12:52:34.624852 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.624787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 12:52:34.625038 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.625019    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nlpmr\""
Jul 28 12:52:34.625290 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.625275    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 12:52:34.650923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.650847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"]
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.684982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-textfile\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6bp\" (UniqueName: \"kubernetes.io/projected/b6a3186d-d911-4511-b390-bf1e454dc0ba-kube-api-access-cl6bp\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a3186d-d911-4511-b390-bf1e454dc0ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-metrics-client-ca\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-root\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685321    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8xkz\" (UniqueName: \"kubernetes.io/projected/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-api-access-q8xkz\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d70a1860-7e59-4c14-819e-c3519bc6a589-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.685985 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6jkn\" (UniqueName: \"kubernetes.io/projected/7c152ab7-1110-4f3e-8498-3b887d27fb64-kube-api-access-c6jkn\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-wtmp\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:34.685604    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:34.685667    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls podName:b6a3186d-d911-4511-b390-bf1e454dc0ba nodeName:}" failed. No retries permitted until 2026-07-28 12:52:35.185646788 +0000 UTC m=+68.502665767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-mzz4b" (UID: "b6a3186d-d911-4511-b390-bf1e454dc0ba") : secret "openshift-state-metrics-tls" not found
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-accelerators-collector-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-sys\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-tls\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.686853 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.685945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a3186d-d911-4511-b390-bf1e454dc0ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.690043 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.689708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.704631 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.704604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6bp\" (UniqueName: \"kubernetes.io/projected/b6a3186d-d911-4511-b390-bf1e454dc0ba-kube-api-access-cl6bp\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:34.786858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.786858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.786858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-metrics-client-ca\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.786858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-root\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.786858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xkz\" (UniqueName: \"kubernetes.io/projected/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-api-access-q8xkz\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.786976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d70a1860-7e59-4c14-819e-c3519bc6a589-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6jkn\" (UniqueName: \"kubernetes.io/projected/7c152ab7-1110-4f3e-8498-3b887d27fb64-kube-api-access-c6jkn\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787053    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-wtmp\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-accelerators-collector-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-sys\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-tls\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787230 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-textfile\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787709 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-textfile\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.787709 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.787676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-root\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.788253 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.788229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.788560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-metrics-client-ca\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.788573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d70a1860-7e59-4c14-819e-c3519bc6a589-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.788872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-sys\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.789125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d70a1860-7e59-4c14-819e-c3519bc6a589-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.789125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-accelerators-collector-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.789445 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.789243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-wtmp\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.791534 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.791445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.792494 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.792455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.793285 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.793259    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.793724 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.793704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c152ab7-1110-4f3e-8498-3b887d27fb64-node-exporter-tls\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.799587 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.799537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6jkn\" (UniqueName: \"kubernetes.io/projected/7c152ab7-1110-4f3e-8498-3b887d27fb64-kube-api-access-c6jkn\") pod \"node-exporter-7dgsl\" (UID: \"7c152ab7-1110-4f3e-8498-3b887d27fb64\") " pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:34.800201 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.800158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xkz\" (UniqueName: \"kubernetes.io/projected/d70a1860-7e59-4c14-819e-c3519bc6a589-kube-api-access-q8xkz\") pod \"kube-state-metrics-56d9f944b4-d6s52\" (UID: \"d70a1860-7e59-4c14-819e-c3519bc6a589\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.932969 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.932924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"
Jul 28 12:52:34.939255 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:34.939228    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7dgsl"
Jul 28 12:52:35.190275 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.190194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:35.193509 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.193457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6a3186d-d911-4511-b390-bf1e454dc0ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mzz4b\" (UID: \"b6a3186d-d911-4511-b390-bf1e454dc0ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:35.416104 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.416073    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"
Jul 28 12:52:35.708790 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.706835    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:52:35.715596 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.715183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.718768 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.718508    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 12:52:35.718768 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.718610    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k4ksc\""
Jul 28 12:52:35.719622 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719591    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 12:52:35.719622 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719614    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 12:52:35.719820 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719641    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 12:52:35.719820 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719691    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 12:52:35.719820 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719794    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 12:52:35.719969 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719821    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 12:52:35.719969 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.719932    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 12:52:35.720067 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.720059    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 12:52:35.726253 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.726225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:52:35.795842 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.795799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796017 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.795848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796017 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.795906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796017 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.795941    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n5pq\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-kube-api-access-6n5pq\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796157 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796025    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796157 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796157 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796307 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796307 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796307 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-out\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796307 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-web-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796507 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.796507 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.796384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897554 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897754 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897830 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897830 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897852    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n5pq\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-kube-api-access-6n5pq\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.897932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.897978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.898003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.898022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-out\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898121 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.898040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-web-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.898728 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:35.898704    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle podName:ae13f4cd-3879-416c-b8db-cd679a36d87c nodeName:}" failed. No retries permitted until 2026-07-28 12:52:36.398678274 +0000 UTC m=+69.715697257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "ae13f4cd-3879-416c-b8db-cd679a36d87c") : configmap references non-existent config key: ca-bundle.crt
Jul 28 12:52:35.899616 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.899588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.900988 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.900965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-web-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.901365 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.901304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.901869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.901844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.903089 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.903065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.903804 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.903782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.904409 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.904358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae13f4cd-3879-416c-b8db-cd679a36d87c-config-out\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.904654 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.904634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.905274 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.905248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.906018 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.905978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae13f4cd-3879-416c-b8db-cd679a36d87c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:35.919191 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:35.919160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n5pq\" (UniqueName: \"kubernetes.io/projected/ae13f4cd-3879-416c-b8db-cd679a36d87c-kube-api-access-6n5pq\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:36.402491 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.402455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:36.403344 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.403323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae13f4cd-3879-416c-b8db-cd679a36d87c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae13f4cd-3879-416c-b8db-cd679a36d87c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:36.574501 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.574462    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"]
Jul 28 12:52:36.577227 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.577198    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.580411 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:36.580380    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-web\" is forbidden: User \"system:node:ip-10-0-132-196.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" type="*v1.Secret"
Jul 28 12:52:36.580640 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:36.580457    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"thanos-querier-5865f4ccb8-vg5r4\" is forbidden: User \"system:node:ip-10-0-132-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-196.ec2.internal' and this object" podUID="e00e332d-7e3b-4005-9d61-3fc1dab4432d" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.580640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.580505    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 12:52:36.580841 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.580825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 12:52:36.580887 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.580837    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1lg7gnkcm8q4a\""
Jul 28 12:52:36.580935 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.580880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 12:52:36.581654 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.581625    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 12:52:36.581771 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.581675    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wl6xf\""
Jul 28 12:52:36.592205 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.592179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"]
Jul 28 12:52:36.630309 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.630274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:52:36.705352 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705352 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705335    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-grpc-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705477    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00e332d-7e3b-4005-9d61-3fc1dab4432d-metrics-client-ca\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.705565 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.705554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpsx8\" (UniqueName: \"kubernetes.io/projected/e00e332d-7e3b-4005-9d61-3fc1dab4432d-kube-api-access-fpsx8\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.806656 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-grpc-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00e332d-7e3b-4005-9d61-3fc1dab4432d-metrics-client-ca\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpsx8\" (UniqueName: \"kubernetes.io/projected/e00e332d-7e3b-4005-9d61-3fc1dab4432d-kube-api-access-fpsx8\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807090 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.806914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.807912 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.807501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00e332d-7e3b-4005-9d61-3fc1dab4432d-metrics-client-ca\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.810071 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.810042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.810186 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.810129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.810242 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.810224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-grpc-tls\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.810281 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.810241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.811503 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.811485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:36.816677 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:36.816651    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpsx8\" (UniqueName: \"kubernetes.io/projected/e00e332d-7e3b-4005-9d61-3fc1dab4432d-kube-api-access-fpsx8\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:37.808290 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:37.808253    2562 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-kube-rbac-proxy-web: failed to sync secret cache: timed out waiting for the condition
Jul 28 12:52:37.808696 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:52:37.808339    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web podName:e00e332d-7e3b-4005-9d61-3fc1dab4432d nodeName:}" failed. No retries permitted until 2026-07-28 12:52:38.308321271 +0000 UTC m=+71.625340242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-thanos-querier-kube-rbac-proxy-web" (UniqueName: "kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web") pod "thanos-querier-5865f4ccb8-vg5r4" (UID: "e00e332d-7e3b-4005-9d61-3fc1dab4432d") : failed to sync secret cache: timed out waiting for the condition
Jul 28 12:52:38.015438 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:38.015396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 12:52:38.321001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:38.320967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:38.329536 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:38.323917    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e00e332d-7e3b-4005-9d61-3fc1dab4432d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5865f4ccb8-vg5r4\" (UID: \"e00e332d-7e3b-4005-9d61-3fc1dab4432d\") " pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:38.389022 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:38.388986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:38.630379 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:38.630308    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5g2pq"
Jul 28 12:52:38.941479 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:38.941450    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c152ab7_1110_4f3e_8498_3b887d27fb64.slice/crio-bb33da261cdad7a5669767bb875d19ba112dc3ddce66458dfe7759c76910cbe7 WatchSource:0}: Error finding container bb33da261cdad7a5669767bb875d19ba112dc3ddce66458dfe7759c76910cbe7: Status 404 returned error can't find the container with id bb33da261cdad7a5669767bb875d19ba112dc3ddce66458dfe7759c76910cbe7
Jul 28 12:52:39.064683 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.062265    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c8c8b8466-2ggg2"]
Jul 28 12:52:39.065921 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.065578    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.070352    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bsb6s\""
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.070587    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.070958    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.071140    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.071355    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1nqq2qcar76eb\""
Jul 28 12:52:39.072923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.071675    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 12:52:39.081556 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.081517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8c8b8466-2ggg2"]
Jul 28 12:52:39.127728 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.127703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-metrics-server-audit-profiles\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.127989 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.127967    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-tls\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.128138 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.128121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-client-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.128343 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.128316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.128485 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.128469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc2b5\" (UniqueName: \"kubernetes.io/projected/3eab24ab-2682-4b75-93af-1707c2bb7eaa-kube-api-access-rc2b5\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.128622 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.128607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.128837 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.128821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3eab24ab-2682-4b75-93af-1707c2bb7eaa-audit-log\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.175367 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.175034    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:52:39.213250 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.213219    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52"]
Jul 28 12:52:39.219392 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:39.219364    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70a1860_7e59_4c14_819e_c3519bc6a589.slice/crio-5cf0c36c20e1d8d17cff822984b84299a0f66b9ec37aea60d970d7ad8d4caf67 WatchSource:0}: Error finding container 5cf0c36c20e1d8d17cff822984b84299a0f66b9ec37aea60d970d7ad8d4caf67: Status 404 returned error can't find the container with id 5cf0c36c20e1d8d17cff822984b84299a0f66b9ec37aea60d970d7ad8d4caf67
Jul 28 12:52:39.230027 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-metrics-server-audit-profiles\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230111 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-tls\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230111 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-client-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230180 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230180 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc2b5\" (UniqueName: \"kubernetes.io/projected/3eab24ab-2682-4b75-93af-1707c2bb7eaa-kube-api-access-rc2b5\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230241 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230241 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3eab24ab-2682-4b75-93af-1707c2bb7eaa-audit-log\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.230581 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.230540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3eab24ab-2682-4b75-93af-1707c2bb7eaa-audit-log\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.231023 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.231001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.231113 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.231007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3eab24ab-2682-4b75-93af-1707c2bb7eaa-metrics-server-audit-profiles\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.232998 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.232977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.233283 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.233261    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-secret-metrics-server-tls\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.233421 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.233402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3eab24ab-2682-4b75-93af-1707c2bb7eaa-client-ca-bundle\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.242932 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.242730    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"]
Jul 28 12:52:39.245256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.245235    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b"]
Jul 28 12:52:39.252612 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.252583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc2b5\" (UniqueName: \"kubernetes.io/projected/3eab24ab-2682-4b75-93af-1707c2bb7eaa-kube-api-access-rc2b5\") pod \"metrics-server-c8c8b8466-2ggg2\" (UID: \"3eab24ab-2682-4b75-93af-1707c2bb7eaa\") " pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.257961 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:39.257939    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a3186d_d911_4511_b390_bf1e454dc0ba.slice/crio-cd8cab8b8e3c684e99afebbf71b5f2e0121f27a06112f9a8fadf385d142006f5 WatchSource:0}: Error finding container cd8cab8b8e3c684e99afebbf71b5f2e0121f27a06112f9a8fadf385d142006f5: Status 404 returned error can't find the container with id cd8cab8b8e3c684e99afebbf71b5f2e0121f27a06112f9a8fadf385d142006f5
Jul 28 12:52:39.286321 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.286299    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:52:39.292908 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.292048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.296663 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.296645    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 12:52:39.296878 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.296858    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 12:52:39.296970 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.296769    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 12:52:39.297032 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.296868    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 12:52:39.297111 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.296664    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 12:52:39.297378 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.297258    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x4m6r\""
Jul 28 12:52:39.297611 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.297565    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 12:52:39.297768 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.297734    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 12:52:39.307136 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.307111    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 12:52:39.307374 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.307360    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:52:39.317425 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.317386    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"]
Jul 28 12:52:39.320125 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.320090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.322995 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.322974    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hnr96\""
Jul 28 12:52:39.323808 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.323793    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 12:52:39.334400 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.334377    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"]
Jul 28 12:52:39.402159 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.402137    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:39.433434 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqz28\" (UniqueName: \"kubernetes.io/projected/12fc22ca-dc5f-43b0-b64f-59a609f23463-kube-api-access-sqz28\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.433434 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znqq5\" (UniqueName: \"kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433589 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433625 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433656 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433631    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433708 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433860 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/12fc22ca-dc5f-43b0-b64f-59a609f23463-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.433860 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.433998 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.433864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534492 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/12fc22ca-dc5f-43b0-b64f-59a609f23463-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.534651 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534970 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.534970 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqz28\" (UniqueName: \"kubernetes.io/projected/12fc22ca-dc5f-43b0-b64f-59a609f23463-kube-api-access-sqz28\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.534970 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.534726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znqq5\" (UniqueName: \"kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.535335 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.535300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.535335 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.535326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.535674 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.535648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.535804 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.535677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.537416 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.537382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/12fc22ca-dc5f-43b0-b64f-59a609f23463-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.537502 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.537382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.537502 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.537442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.544463 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.544438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8c8b8466-2ggg2"]
Jul 28 12:52:39.546260 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:39.546238    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eab24ab_2682_4b75_93af_1707c2bb7eaa.slice/crio-c284ae51459b277aa2e0083d4778b9b4cae6523faf21722f56c407a58d8fd1b0 WatchSource:0}: Error finding container c284ae51459b277aa2e0083d4778b9b4cae6523faf21722f56c407a58d8fd1b0: Status 404 returned error can't find the container with id c284ae51459b277aa2e0083d4778b9b4cae6523faf21722f56c407a58d8fd1b0
Jul 28 12:52:39.549109 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.549086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znqq5\" (UniqueName: \"kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5\") pod \"console-84c88889c8-c7bjt\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") " pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.549179 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.549095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqz28\" (UniqueName: \"kubernetes.io/projected/12fc22ca-dc5f-43b0-b64f-59a609f23463-kube-api-access-sqz28\") pod \"monitoring-plugin-68b7fc9574-gs855\" (UID: \"12fc22ca-dc5f-43b0-b64f-59a609f23463\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.608577 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.608550    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:39.649104 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.649075    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:39.677464 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.677430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7dgsl" event={"ID":"7c152ab7-1110-4f3e-8498-3b887d27fb64","Type":"ContainerStarted","Data":"bb33da261cdad7a5669767bb875d19ba112dc3ddce66458dfe7759c76910cbe7"}
Jul 28 12:52:39.679161 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.679131    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bhz6" event={"ID":"021805c8-b3fb-4e66-ba52-361313f279d5","Type":"ContainerStarted","Data":"20f7c587d3abce7c3078d58d025a47e54cbe686e3fbd81210b8fb8670e1bc733"}
Jul 28 12:52:39.679281 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.679253    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:52:39.680449 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.680384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"80e2bd7d34d2ad744128aeb42b8d54046e260fd0fa897f0152fef7ba2e2df54d"}
Jul 28 12:52:39.682363 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.682339    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"600c92356d2ecfc80f9bfec4f7c053d3f0c31b28402aa4797dfb17aa0666e5d0"}
Jul 28 12:52:39.684041 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.683982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52" event={"ID":"d70a1860-7e59-4c14-819e-c3519bc6a589","Type":"ContainerStarted","Data":"5cf0c36c20e1d8d17cff822984b84299a0f66b9ec37aea60d970d7ad8d4caf67"}
Jul 28 12:52:39.686485 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.686445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fv5kv" event={"ID":"fe4d3851-5860-4c58-9921-741f4a99e615","Type":"ContainerStarted","Data":"24650afc395836ad1330f94abc4ce3efdc1e813d42008ebb6b4c94ec5dcfc8a8"}
Jul 28 12:52:39.688864 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.688841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v6r2" event={"ID":"2fdfd184-4960-4e0e-975b-e70415a99242","Type":"ContainerStarted","Data":"f1be69c6b50029469d00dd93e5aea962cb2d89c16b0271b477c8f83e8d5e3375"}
Jul 28 12:52:39.688961 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.688885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v6r2" event={"ID":"2fdfd184-4960-4e0e-975b-e70415a99242","Type":"ContainerStarted","Data":"b98e2b210a19128ff2a30e05ea574a36d586c94af28bfd6f38520c24d5f14990"}
Jul 28 12:52:39.690796 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.690771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2" event={"ID":"3eab24ab-2682-4b75-93af-1707c2bb7eaa","Type":"ContainerStarted","Data":"c284ae51459b277aa2e0083d4778b9b4cae6523faf21722f56c407a58d8fd1b0"}
Jul 28 12:52:39.693789 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.693221    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mplhd" event={"ID":"32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e","Type":"ContainerStarted","Data":"0e0f381e5946f15b20de5e1fb377e308bc73cf7e0554f4d97f444e4847cba105"}
Jul 28 12:52:39.695869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.695722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b" event={"ID":"b6a3186d-d911-4511-b390-bf1e454dc0ba","Type":"ContainerStarted","Data":"5348c6909515187b77d21c1551be2faed529a1134c2097aa5555922b5a9f162a"}
Jul 28 12:52:39.695869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.695789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b" event={"ID":"b6a3186d-d911-4511-b390-bf1e454dc0ba","Type":"ContainerStarted","Data":"a11800c20bf6a82feb80eee65ec7f21b54e673e7c851031529e341ca79226e65"}
Jul 28 12:52:39.695869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.695808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b" event={"ID":"b6a3186d-d911-4511-b390-bf1e454dc0ba","Type":"ContainerStarted","Data":"cd8cab8b8e3c684e99afebbf71b5f2e0121f27a06112f9a8fadf385d142006f5"}
Jul 28 12:52:39.702413 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.702358    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2bhz6" podStartSLOduration=65.990236351 podStartE2EDuration="1m11.702339697s" totalImagesPullingTime="5.712103346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:39.699253402 +0000 UTC m=+73.016272397" watchObservedRunningTime="2026-07-28 12:52:39.702339697 +0000 UTC m=+73.019358690"
Jul 28 12:52:39.725980 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.725915    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fv5kv" podStartSLOduration=66.106363089 podStartE2EDuration="1m11.725892427s" totalImagesPullingTime="5.619529338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:39.723110805 +0000 UTC m=+73.040129815" watchObservedRunningTime="2026-07-28 12:52:39.725892427 +0000 UTC m=+73.042911398"
Jul 28 12:52:39.749950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.749892    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mplhd" podStartSLOduration=2.113317169 podStartE2EDuration="18.749869594s" totalImagesPullingTime="16.636552425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:52:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:39.748450641 +0000 UTC m=+73.065469660" watchObservedRunningTime="2026-07-28 12:52:39.749869594 +0000 UTC m=+73.066888585"
Jul 28 12:52:39.766721 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.766684    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:52:39.767568 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.767519    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2v6r2" podStartSLOduration=66.105830999 podStartE2EDuration="1m11.767501536s" totalImagesPullingTime="5.661670537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:39.766171976 +0000 UTC m=+73.083190966" watchObservedRunningTime="2026-07-28 12:52:39.767501536 +0000 UTC m=+73.084520525"
Jul 28 12:52:39.768929 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:39.768889    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954cb79f_016c_4a85_9d62_fa813ef5a3a1.slice/crio-bf837fa70e825ca325ede60200b763957dddd2ff7353bf8f52a843c0af8a1455 WatchSource:0}: Error finding container bf837fa70e825ca325ede60200b763957dddd2ff7353bf8f52a843c0af8a1455: Status 404 returned error can't find the container with id bf837fa70e825ca325ede60200b763957dddd2ff7353bf8f52a843c0af8a1455
Jul 28 12:52:39.815252 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:39.815215    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"]
Jul 28 12:52:39.820615 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:39.820585    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fc22ca_dc5f_43b0_b64f_59a609f23463.slice/crio-a070af569b4fae09c5642a87d06f59612a84bb388956cebb69f04bc7729a6039 WatchSource:0}: Error finding container a070af569b4fae09c5642a87d06f59612a84bb388956cebb69f04bc7729a6039: Status 404 returned error can't find the container with id a070af569b4fae09c5642a87d06f59612a84bb388956cebb69f04bc7729a6039
Jul 28 12:52:40.703413 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.703321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855" event={"ID":"12fc22ca-dc5f-43b0-b64f-59a609f23463","Type":"ContainerStarted","Data":"a070af569b4fae09c5642a87d06f59612a84bb388956cebb69f04bc7729a6039"}
Jul 28 12:52:40.707852 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.707085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c88889c8-c7bjt" event={"ID":"954cb79f-016c-4a85-9d62-fa813ef5a3a1","Type":"ContainerStarted","Data":"bf837fa70e825ca325ede60200b763957dddd2ff7353bf8f52a843c0af8a1455"}
Jul 28 12:52:40.898359 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.898037    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 12:52:40.906233 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.904528    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:40.908820 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.908790    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 12:52:40.909285 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.909264    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hsw2n\""
Jul 28 12:52:40.909502 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.909484    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e5u23hnicq7ps\""
Jul 28 12:52:40.909670 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.909652    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 12:52:40.910045 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.910025    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 12:52:40.910268 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.910251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 12:52:40.912023 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.911997    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 12:52:40.912782 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912240    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 12:52:40.912782 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912415    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 12:52:40.912782 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912505    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 12:52:40.912782 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 12:52:40.913024 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912840    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 12:52:40.913024 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.912979    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 12:52:40.916869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.915894    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 12:52:40.916869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.916157    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 12:52:40.916869 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.916378    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 12:52:40.921902 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:40.920575    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050258    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.050964 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.051895 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.051895 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.051895 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.051895 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.050717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njpxj\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-kube-api-access-njpxj\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.151493 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.151364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.152364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njpxj\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-kube-api-access-njpxj\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.156506 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.153725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.154117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.154978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.160765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.161368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.163298 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.163150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164189 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.163541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164189 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.163685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164189 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.163825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba756922-f0ee-4333-b34d-bff9ed99faed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164189 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.163826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164189 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.164143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.164513 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.164483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.165198 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.165032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.165198 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.165141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.165375 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.165293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.167495 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.166104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.167495 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.166714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba756922-f0ee-4333-b34d-bff9ed99faed-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.167495 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.167104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.170008 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.169978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.172559 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.172538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba756922-f0ee-4333-b34d-bff9ed99faed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.177794 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.177735    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njpxj\" (UniqueName: \"kubernetes.io/projected/ba756922-f0ee-4333-b34d-bff9ed99faed-kube-api-access-njpxj\") pod \"prometheus-k8s-0\" (UID: \"ba756922-f0ee-4333-b34d-bff9ed99faed\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:41.224002 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:41.223964    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:42.682547 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:42.682512    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 12:52:42.819169 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:52:42.819139    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba756922_f0ee_4333_b34d_bff9ed99faed.slice/crio-c937abc3585b92e7b9f9203491202555450f1709d4d8f5ecca0554f976cce84b WatchSource:0}: Error finding container c937abc3585b92e7b9f9203491202555450f1709d4d8f5ecca0554f976cce84b: Status 404 returned error can't find the container with id c937abc3585b92e7b9f9203491202555450f1709d4d8f5ecca0554f976cce84b
Jul 28 12:52:43.722165 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:43.722124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"c937abc3585b92e7b9f9203491202555450f1709d4d8f5ecca0554f976cce84b"}
Jul 28 12:52:44.730780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.730664    2562 generic.go:356] "Generic (PLEG): container finished" podID="ba756922-f0ee-4333-b34d-bff9ed99faed" containerID="c1d00da7a87434a105b910a1a15401465b53163276bfe0276c0d4fd65946534d" exitCode=0
Jul 28 12:52:44.730780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.730773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerDied","Data":"c1d00da7a87434a105b910a1a15401465b53163276bfe0276c0d4fd65946534d"}
Jul 28 12:52:44.736162 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.736068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"7d8ff9d0b52b816f0a87f47e5c8740867aabe3a7996b4d21f9132f59eb3907ed"}
Jul 28 12:52:44.736162 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.736148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"a4396b92be228c9cdd6b0814d9537eeaaf6b0df806574aaf2ba961a5956d44be"}
Jul 28 12:52:44.736318 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.736195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"92335eeb83c8a7d620df212271422fc30927adf91efd0d37764beb7b989c5c01"}
Jul 28 12:52:44.738979 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.738784    2562 generic.go:356] "Generic (PLEG): container finished" podID="ae13f4cd-3879-416c-b8db-cd679a36d87c" containerID="8d61300e19521aaa1926aa3a065ae0fcd7f9e2b3ca201996d160aaff27e1a71c" exitCode=0
Jul 28 12:52:44.738979 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.738861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerDied","Data":"8d61300e19521aaa1926aa3a065ae0fcd7f9e2b3ca201996d160aaff27e1a71c"}
Jul 28 12:52:44.745536 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.745481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52" event={"ID":"d70a1860-7e59-4c14-819e-c3519bc6a589","Type":"ContainerStarted","Data":"dd194c603568c36b30c9bd5208afbe74b86c3703c565e9ba76f716d9ef8684b7"}
Jul 28 12:52:44.745641 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.745602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52" event={"ID":"d70a1860-7e59-4c14-819e-c3519bc6a589","Type":"ContainerStarted","Data":"6ffab3ea2347bfe5c615c440a4804b1bfa5f29cf7c5530d0f57b8b4d2cc095ab"}
Jul 28 12:52:44.745641 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.745633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52" event={"ID":"d70a1860-7e59-4c14-819e-c3519bc6a589","Type":"ContainerStarted","Data":"910133a0fba1ef45c11159046df5bab0c8f327a232f3bee9b1971de31d4df6f6"}
Jul 28 12:52:44.748138 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.748088    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2" event={"ID":"3eab24ab-2682-4b75-93af-1707c2bb7eaa","Type":"ContainerStarted","Data":"c4f88c915e9a6bf797ecd165d0347971b602233dda0a6ebf2e27535d497a09a5"}
Jul 28 12:52:44.749708 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.749682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855" event={"ID":"12fc22ca-dc5f-43b0-b64f-59a609f23463","Type":"ContainerStarted","Data":"9658277eca0ce6d6a811ce2a4aed57ae4fdba14057d923b4968f305da0726567"}
Jul 28 12:52:44.750780 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.750727    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:44.752412 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.752353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c88889c8-c7bjt" event={"ID":"954cb79f-016c-4a85-9d62-fa813ef5a3a1","Type":"ContainerStarted","Data":"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"}
Jul 28 12:52:44.758116 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.758069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b" event={"ID":"b6a3186d-d911-4511-b390-bf1e454dc0ba","Type":"ContainerStarted","Data":"0337218b260f5e65a61acf9f0407ce21234925ca6dde92008256101cf2db1331"}
Jul 28 12:52:44.758512 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.758498    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855"
Jul 28 12:52:44.760681 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.760657    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c152ab7-1110-4f3e-8498-3b887d27fb64" containerID="b202920a1e17b705e80dae4851a0688463c96fd88f49969a98ae86b2134c29f5" exitCode=0
Jul 28 12:52:44.760929 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.760697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7dgsl" event={"ID":"7c152ab7-1110-4f3e-8498-3b887d27fb64","Type":"ContainerDied","Data":"b202920a1e17b705e80dae4851a0688463c96fd88f49969a98ae86b2134c29f5"}
Jul 28 12:52:44.836402 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.836340    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mzz4b" podStartSLOduration=5.946970113 podStartE2EDuration="10.836324755s" totalImagesPullingTime="4.889354642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:44.835436615 +0000 UTC m=+78.152455605" watchObservedRunningTime="2026-07-28 12:52:44.836324755 +0000 UTC m=+78.153343745"
Jul 28 12:52:44.892492 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.891784    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84c88889c8-c7bjt" podStartSLOduration=1.297261778 podStartE2EDuration="5.891764002s" totalImagesPullingTime="4.594502224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:44.889373357 +0000 UTC m=+78.206392348" watchObservedRunningTime="2026-07-28 12:52:44.891764002 +0000 UTC m=+78.208782973"
Jul 28 12:52:44.928685 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.928624    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-d6s52" podStartSLOduration=7.635176921 podStartE2EDuration="10.92860112s" totalImagesPullingTime="3.293424199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:44.927549736 +0000 UTC m=+78.244568726" watchObservedRunningTime="2026-07-28 12:52:44.92860112 +0000 UTC m=+78.245620111"
Jul 28 12:52:44.958351 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:44.958292    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gs855" podStartSLOduration=1.486468521 podStartE2EDuration="5.958275825s" totalImagesPullingTime="4.471807304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:44.956426132 +0000 UTC m=+78.273445123" watchObservedRunningTime="2026-07-28 12:52:44.958275825 +0000 UTC m=+78.275294794"
Jul 28 12:52:45.038349 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.038294    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2" podStartSLOduration=2.390305927 podStartE2EDuration="6.038261009s" totalImagesPullingTime="3.647955082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:45.037217833 +0000 UTC m=+78.354236848" watchObservedRunningTime="2026-07-28 12:52:45.038261009 +0000 UTC m=+78.355279998"
Jul 28 12:52:45.780956 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.780904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7dgsl" event={"ID":"7c152ab7-1110-4f3e-8498-3b887d27fb64","Type":"ContainerStarted","Data":"6bc9a0d7fae2d1a00bee4ebc3949730c54d582c5e5c1aac8a01e1ba40268ab16"}
Jul 28 12:52:45.781348 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.781021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7dgsl" event={"ID":"7c152ab7-1110-4f3e-8498-3b887d27fb64","Type":"ContainerStarted","Data":"809f327ea2c9a39f4a10fda05f6cd454e57aa9235befef3bea181f69ebf95475"}
Jul 28 12:52:45.789317 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.788801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"3e24c560cdcffbdb356b48965c02663baa5f1d36eb71e6c3bac595975b341b5a"}
Jul 28 12:52:45.789317 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.788876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"1d0ba8715843b90b5a6d4c42f24938c8abbb07f5e40e0659ef6bb4709942d257"}
Jul 28 12:52:45.805950 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:45.805067    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7dgsl" podStartSLOduration=8.089339925 podStartE2EDuration="11.805044325s" totalImagesPullingTime="3.567537316s" totalInitContainerRuntime="148.167084ms" isStatefulPod=true podCreationTimestamp="2026-07-28 12:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:45.803429303 +0000 UTC m=+79.120448295" watchObservedRunningTime="2026-07-28 12:52:45.805044325 +0000 UTC m=+79.122063317"
Jul 28 12:52:46.795563 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.795478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" event={"ID":"e00e332d-7e3b-4005-9d61-3fc1dab4432d","Type":"ContainerStarted","Data":"1738f0fe38d70679974793f1aaaf225daf1cd46bedbca45795fb6727445e9b34"}
Jul 28 12:52:46.796052 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.795633    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"9f033003b034000f213ef04e7bb15ce1510bc802204c0bbc1f38640c290b354d"}
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"4e6fb41ee128cbf4b9b65560629ddf24456eeb4a9827ceb5c12fcf1b65ec4b62"}
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"36efc8b8d78da2554f1eacbb8703f15a858d2d62cd116c11d7e30e9fcf3fd680"}
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800349    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"df3a5aa51e7e29490fc883dcb3d9db87138f9fd8bac0f58107bd07a07ddfeb11"}
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800387    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"486600d2e1205b596de628452ec19f2680352c437f7328cadb66ade5698e854b"}
Jul 28 12:52:46.800800 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.800427    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae13f4cd-3879-416c-b8db-cd679a36d87c","Type":"ContainerStarted","Data":"d6752450a90a7dc442f15897ea37de85d6bc44dde27b9e0911829db1864de001"}
Jul 28 12:52:46.822246 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.820610    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4" podStartSLOduration=6.388750015 podStartE2EDuration="10.820590485s" totalImagesPullingTime="4.43184047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:46.820152706 +0000 UTC m=+80.137171737" watchObservedRunningTime="2026-07-28 12:52:46.820590485 +0000 UTC m=+80.137609476"
Jul 28 12:52:46.851184 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:46.851126    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.958399582 podStartE2EDuration="11.851103635s" totalImagesPullingTime="4.858470595s" totalInitContainerRuntime="34.233458ms" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:46.849593698 +0000 UTC m=+80.166612688" watchObservedRunningTime="2026-07-28 12:52:46.851103635 +0000 UTC m=+80.168122630"
Jul 28 12:52:48.814257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:48.814104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"f780c7d59b3a0b8d890c34345115de528de6f826104f047da6055cbe7bbdb31c"}
Jul 28 12:52:48.814257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:48.814169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"06c58cf4ce41318b959507359f81b882d0b8817eb0df9b3bcdc9771d7f852b12"}
Jul 28 12:52:48.814257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:48.814196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"e7630e00f353e1f4cbe009d19f621bd2be4b75b04dd50f68213c9f05d0230fb8"}
Jul 28 12:52:48.814257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:48.814219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"87bb968d7d7b41999b074e1b9ee2805d66bffa3c4ab2025983b1e1e1205a3838"}
Jul 28 12:52:48.814257 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:48.814242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"d7516ccceac442194ba7b85f7b41f804bbd7dc36913b071b93f83444f2841364"}
Jul 28 12:52:49.608902 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.608860    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:49.609164 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.608931    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:49.613581 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.613558    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:49.824152 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.824114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba756922-f0ee-4333-b34d-bff9ed99faed","Type":"ContainerStarted","Data":"5d7be2f9f787d4ab5f705a2c96419d12235b3ea2b00afd4124f38282ce677500"}
Jul 28 12:52:49.827982 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.827960    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:52:49.861369 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:49.861243    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.09975271 podStartE2EDuration="9.861227662s" totalImagesPullingTime="3.708576444s" totalInitContainerRuntime="52.898508ms" isStatefulPod=false podCreationTimestamp="2026-07-28 12:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:52:49.859225057 +0000 UTC m=+83.176244065" watchObservedRunningTime="2026-07-28 12:52:49.861227662 +0000 UTC m=+83.178246652"
Jul 28 12:52:51.225067 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:51.225026    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:52:52.810552 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:52.810524    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5865f4ccb8-vg5r4"
Jul 28 12:52:59.403116 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:59.403081    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:52:59.403572 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:52:59.403149    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:53:10.709823 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:10.709792    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2bhz6"
Jul 28 12:53:19.408229 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:19.408202    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:53:19.412303 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:19.412269    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c8c8b8466-2ggg2"
Jul 28 12:53:41.224909 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:41.224871    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:53:41.244422 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:41.244389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:53:42.069388 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:53:42.069354    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 12:54:21.463201 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:21.463121    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:54:46.482905 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.482858    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84c88889c8-c7bjt" podUID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" containerName="console" containerID="cri-o://170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386" gracePeriod=15
Jul 28 12:54:46.715510 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.715486    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:54:46.742292 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742268    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742442 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742306    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742442 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742332    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znqq5\" (UniqueName: \"kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742442 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742372    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742590 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742500    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742590 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742557    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742690 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742659    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config\") pod \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\" (UID: \"954cb79f-016c-4a85-9d62-fa813ef5a3a1\") "
Jul 28 12:54:46.742768 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca" (OuterVolumeSpecName: "service-ca") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:54:46.742861 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742833    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:54:46.742861 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742844    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:54:46.742963 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.742925    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config" (OuterVolumeSpecName: "console-config") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:54:46.743131 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.743111    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.743131 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.743133    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-oauth-serving-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.743300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.743147    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-trusted-ca-bundle\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.743300 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.743160    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-config\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.745444 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.745407    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:54:46.745571 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.745545    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:54:46.745635 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.745578    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5" (OuterVolumeSpecName: "kube-api-access-znqq5") pod "954cb79f-016c-4a85-9d62-fa813ef5a3a1" (UID: "954cb79f-016c-4a85-9d62-fa813ef5a3a1"). InnerVolumeSpecName "kube-api-access-znqq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:54:46.843888 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.843854    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-oauth-config\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.843888 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.843880    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znqq5\" (UniqueName: \"kubernetes.io/projected/954cb79f-016c-4a85-9d62-fa813ef5a3a1-kube-api-access-znqq5\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:46.843888 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:46.843894    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/954cb79f-016c-4a85-9d62-fa813ef5a3a1-console-serving-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 12:54:47.311109 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.311071    2562 generic.go:356] "Generic (PLEG): container finished" podID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" containerID="170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386" exitCode=0
Jul 28 12:54:47.311256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.311127    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c88889c8-c7bjt"
Jul 28 12:54:47.311256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.311134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c88889c8-c7bjt" event={"ID":"954cb79f-016c-4a85-9d62-fa813ef5a3a1","Type":"ContainerDied","Data":"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"}
Jul 28 12:54:47.311256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.311183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c88889c8-c7bjt" event={"ID":"954cb79f-016c-4a85-9d62-fa813ef5a3a1","Type":"ContainerDied","Data":"bf837fa70e825ca325ede60200b763957dddd2ff7353bf8f52a843c0af8a1455"}
Jul 28 12:54:47.311256 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.311208    2562 scope.go:117] "RemoveContainer" containerID="170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"
Jul 28 12:54:47.319784 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.319758    2562 scope.go:117] "RemoveContainer" containerID="170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"
Jul 28 12:54:47.320075 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:54:47.320048    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386\": container with ID starting with 170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386 not found: ID does not exist" containerID="170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"
Jul 28 12:54:47.320141 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.320088    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386"} err="failed to get container status \"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386\": rpc error: code = NotFound desc = could not find container \"170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386\": container with ID starting with 170a9bd304c9e04216114135d53e90c492161d7fa6ad0f7643cd4ceb7545e386 not found: ID does not exist"
Jul 28 12:54:47.332030 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.331960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:54:47.333858 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:47.333822    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84c88889c8-c7bjt"]
Jul 28 12:54:49.251802 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:54:49.251768    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" path="/var/lib/kubelet/pods/954cb79f-016c-4a85-9d62-fa813ef5a3a1/volumes"
Jul 28 12:56:27.069724 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:27.069698    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 12:56:44.174854 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.174814    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"]
Jul 28 12:56:44.176349 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.175139    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" containerName="console"
Jul 28 12:56:44.176349 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.175149    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" containerName="console"
Jul 28 12:56:44.176349 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.175206    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="954cb79f-016c-4a85-9d62-fa813ef5a3a1" containerName="console"
Jul 28 12:56:44.176788 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.176774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.179248 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.179225    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 12:56:44.179367 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.179346    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8tpqb\""
Jul 28 12:56:44.180312 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.180295    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 12:56:44.189391 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.189369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"]
Jul 28 12:56:44.338038 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.337992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88ks\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-kube-api-access-f88ks\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.338331 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.338307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.439216 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.439120    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.439216 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.439218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88ks\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-kube-api-access-f88ks\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.448254 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.448220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.449607 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.449583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88ks\" (UniqueName: \"kubernetes.io/projected/286f41f8-81d2-41d3-8215-04e4d7d57f19-kube-api-access-f88ks\") pod \"cert-manager-cainjector-559cdd8c87-5fk76\" (UID: \"286f41f8-81d2-41d3-8215-04e4d7d57f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.496640 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.496605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"
Jul 28 12:56:44.623568 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.623544    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5fk76"]
Jul 28 12:56:44.626020 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:56:44.625989    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod286f41f8_81d2_41d3_8215_04e4d7d57f19.slice/crio-7cac57bc213264a9ad0d81ca6d3474caded184f9254f474ca8e3b9f6b102357b WatchSource:0}: Error finding container 7cac57bc213264a9ad0d81ca6d3474caded184f9254f474ca8e3b9f6b102357b: Status 404 returned error can't find the container with id 7cac57bc213264a9ad0d81ca6d3474caded184f9254f474ca8e3b9f6b102357b
Jul 28 12:56:44.628064 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.628046    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:56:44.762579 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:44.762540    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76" event={"ID":"286f41f8-81d2-41d3-8215-04e4d7d57f19","Type":"ContainerStarted","Data":"7cac57bc213264a9ad0d81ca6d3474caded184f9254f474ca8e3b9f6b102357b"}
Jul 28 12:56:48.782448 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:48.782415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76" event={"ID":"286f41f8-81d2-41d3-8215-04e4d7d57f19","Type":"ContainerStarted","Data":"85fa1920de7377ca41f93a5b0172e5f59e58a2e585af191d40ab1ef9333b20fe"}
Jul 28 12:56:48.802695 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:48.802644    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5fk76" podStartSLOduration=1.5127582529999999 podStartE2EDuration="4.802626985s" totalImagesPullingTime="3.289868732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:56:48.801275105 +0000 UTC m=+322.118294095" watchObservedRunningTime="2026-07-28 12:56:48.802626985 +0000 UTC m=+322.119645972"
Jul 28 12:56:54.736316 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.736279    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ph9dr"]
Jul 28 12:56:54.739674 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.739658    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:54.742878 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.742846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dh8d9\""
Jul 28 12:56:54.755049 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.755023    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ph9dr"]
Jul 28 12:56:54.930812 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.930774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-bound-sa-token\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:54.930812 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:54.930816    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52psw\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-kube-api-access-52psw\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.032011 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.031976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-bound-sa-token\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.032011 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.032012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52psw\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-kube-api-access-52psw\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.042557 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.042535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52psw\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-kube-api-access-52psw\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.043012 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.042994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98aec30-b8a4-42f8-898a-eb42d410621d-bound-sa-token\") pod \"cert-manager-55b4d954d5-ph9dr\" (UID: \"f98aec30-b8a4-42f8-898a-eb42d410621d\") " pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.048940 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.048914    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ph9dr"
Jul 28 12:56:55.179227 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.179203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ph9dr"]
Jul 28 12:56:55.181857 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:56:55.181830    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf98aec30_b8a4_42f8_898a_eb42d410621d.slice/crio-b1895b402c7c84900d1462c6e7bb14d8ac478c9006c1413e2c15c1be3de3f620 WatchSource:0}: Error finding container b1895b402c7c84900d1462c6e7bb14d8ac478c9006c1413e2c15c1be3de3f620: Status 404 returned error can't find the container with id b1895b402c7c84900d1462c6e7bb14d8ac478c9006c1413e2c15c1be3de3f620
Jul 28 12:56:55.810726 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.810696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ph9dr" event={"ID":"f98aec30-b8a4-42f8-898a-eb42d410621d","Type":"ContainerStarted","Data":"fd48129257baa2f6af8160d32a6dbace15a2ed58c1386fae6a3f12c0af0cc9c6"}
Jul 28 12:56:55.810726 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.810732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ph9dr" event={"ID":"f98aec30-b8a4-42f8-898a-eb42d410621d","Type":"ContainerStarted","Data":"b1895b402c7c84900d1462c6e7bb14d8ac478c9006c1413e2c15c1be3de3f620"}
Jul 28 12:56:55.849973 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:56:55.849915    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ph9dr" podStartSLOduration=1.849899249 podStartE2EDuration="1.849899249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:56:55.848414104 +0000 UTC m=+329.165433091" watchObservedRunningTime="2026-07-28 12:56:55.849899249 +0000 UTC m=+329.166918238"
Jul 28 12:57:20.928197 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.928115    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"]
Jul 28 12:57:20.931499 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.931463    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:20.934429 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.934404    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 12:57:20.935213 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.935197    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 12:57:20.935319 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.935244    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 12:57:20.935923 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.935906    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n5pcj\""
Jul 28 12:57:20.936144 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.936131    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 12:57:20.949556 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:20.949529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"]
Jul 28 12:57:21.051686 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.051648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-tmp\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.051896 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.051715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.051896 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.051804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-manifests\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.051896 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.051877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c865p\" (UniqueName: \"kubernetes.io/projected/85cdd0cd-74d2-4e4c-926b-f86352a7b123-kube-api-access-c865p\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.052012 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.051898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.152922 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.152871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.153122 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.152986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-manifests\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.153122 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.153064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c865p\" (UniqueName: \"kubernetes.io/projected/85cdd0cd-74d2-4e4c-926b-f86352a7b123-kube-api-access-c865p\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.153122 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.153086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.153122 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.153108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-tmp\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.153439 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.153419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-manifests\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.155345 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.155321    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cdd0cd-74d2-4e4c-926b-f86352a7b123-tmp\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.155461 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.155443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.155642 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.155622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85cdd0cd-74d2-4e4c-926b-f86352a7b123-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.162229 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.162206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c865p\" (UniqueName: \"kubernetes.io/projected/85cdd0cd-74d2-4e4c-926b-f86352a7b123-kube-api-access-c865p\") pod \"opendatahub-operator-controller-manager-64bb544cf8-8sjpf\" (UID: \"85cdd0cd-74d2-4e4c-926b-f86352a7b123\") " pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.243887 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.243865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:21.405264 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.405236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"]
Jul 28 12:57:21.406065 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:57:21.406035    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cdd0cd_74d2_4e4c_926b_f86352a7b123.slice/crio-2887bafdca98da66e428cfa2350e90b3aaf394dc35df1f7fbb3d5383389aa72c WatchSource:0}: Error finding container 2887bafdca98da66e428cfa2350e90b3aaf394dc35df1f7fbb3d5383389aa72c: Status 404 returned error can't find the container with id 2887bafdca98da66e428cfa2350e90b3aaf394dc35df1f7fbb3d5383389aa72c
Jul 28 12:57:21.518153 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.518066    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"]
Jul 28 12:57:21.522451 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.522432    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.525379 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.525356    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 12:57:21.525630 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.525612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h4ctg\""
Jul 28 12:57:21.525754 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.525650    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 12:57:21.525754 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.525650    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:57:21.525854 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.525650    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 12:57:21.526543 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.526530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 12:57:21.535909 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.535887    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"]
Jul 28 12:57:21.657544 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.657498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d54f5603-b633-45f5-94c2-a99080eed723-manager-config\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.657722 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.657566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.657722 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.657596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjptm\" (UniqueName: \"kubernetes.io/projected/d54f5603-b633-45f5-94c2-a99080eed723-kube-api-access-jjptm\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.657722 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.657641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-metrics-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.758108 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.758068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d54f5603-b633-45f5-94c2-a99080eed723-manager-config\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.758291 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.758128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.758291 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.758158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjptm\" (UniqueName: \"kubernetes.io/projected/d54f5603-b633-45f5-94c2-a99080eed723-kube-api-access-jjptm\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.758291 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.758202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-metrics-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.758710 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.758688    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d54f5603-b633-45f5-94c2-a99080eed723-manager-config\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.760764 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.760728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.760845 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.760799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d54f5603-b633-45f5-94c2-a99080eed723-metrics-cert\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.773960 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.773906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjptm\" (UniqueName: \"kubernetes.io/projected/d54f5603-b633-45f5-94c2-a99080eed723-kube-api-access-jjptm\") pod \"lws-controller-manager-5fcd6f4d74-ct49f\" (UID: \"d54f5603-b633-45f5-94c2-a99080eed723\") " pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.832977 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.832937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:21.915947 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.915910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf" event={"ID":"85cdd0cd-74d2-4e4c-926b-f86352a7b123","Type":"ContainerStarted","Data":"2887bafdca98da66e428cfa2350e90b3aaf394dc35df1f7fbb3d5383389aa72c"}
Jul 28 12:57:21.975883 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:21.975858    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"]
Jul 28 12:57:21.977885 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:57:21.977857    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54f5603_b633_45f5_94c2_a99080eed723.slice/crio-471d600372a79e914891f14d48d86d73cf9f0ed956ff8e30f29895e4df5d6d1c WatchSource:0}: Error finding container 471d600372a79e914891f14d48d86d73cf9f0ed956ff8e30f29895e4df5d6d1c: Status 404 returned error can't find the container with id 471d600372a79e914891f14d48d86d73cf9f0ed956ff8e30f29895e4df5d6d1c
Jul 28 12:57:22.926519 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:22.926475    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f" event={"ID":"d54f5603-b633-45f5-94c2-a99080eed723","Type":"ContainerStarted","Data":"471d600372a79e914891f14d48d86d73cf9f0ed956ff8e30f29895e4df5d6d1c"}
Jul 28 12:57:26.952549 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:26.952515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f" event={"ID":"d54f5603-b633-45f5-94c2-a99080eed723","Type":"ContainerStarted","Data":"5fe0290798501a124f9683b61ebc9438bb3fed1d055efcb2b54bbf96d6ebbb84"}
Jul 28 12:57:26.953030 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:26.952573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:26.954125 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:26.954106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf" event={"ID":"85cdd0cd-74d2-4e4c-926b-f86352a7b123","Type":"ContainerStarted","Data":"f975925a27e58f6df684f31461fd53f1e30b9016846383063ea60337cdd0bc31"}
Jul 28 12:57:26.984396 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:26.984347    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f" podStartSLOduration=1.847867113 podStartE2EDuration="5.984334361s" totalImagesPullingTime="4.136467248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:57:26.982882125 +0000 UTC m=+360.299901115" watchObservedRunningTime="2026-07-28 12:57:26.984334361 +0000 UTC m=+360.301353350"
Jul 28 12:57:27.961178 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:27.961143    2562 generic.go:356] "Generic (PLEG): container finished" podID="85cdd0cd-74d2-4e4c-926b-f86352a7b123" containerID="f975925a27e58f6df684f31461fd53f1e30b9016846383063ea60337cdd0bc31" exitCode=0
Jul 28 12:57:27.961557 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:27.961230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf" event={"ID":"85cdd0cd-74d2-4e4c-926b-f86352a7b123","Type":"ContainerDied","Data":"f975925a27e58f6df684f31461fd53f1e30b9016846383063ea60337cdd0bc31"}
Jul 28 12:57:28.967598 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:28.967563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf" event={"ID":"85cdd0cd-74d2-4e4c-926b-f86352a7b123","Type":"ContainerStarted","Data":"0444502c4e62c3576667191f30bbfb68b5c35308367158f1484d93b0c8a235b1"}
Jul 28 12:57:28.968001 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:28.967701    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:28.997715 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:28.997661    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf" podStartSLOduration=2.790238577 podStartE2EDuration="8.997644895s" totalImagesPullingTime="5.184185409s" totalInitContainerRuntime="1.023220909s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:57:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:57:28.99572447 +0000 UTC m=+362.312743461" watchObservedRunningTime="2026-07-28 12:57:28.997644895 +0000 UTC m=+362.314663885"
Jul 28 12:57:37.963768 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:37.963712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5fcd6f4d74-ct49f"
Jul 28 12:57:39.974237 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:39.974209    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64bb544cf8-8sjpf"
Jul 28 12:57:45.252261 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.252232    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7897958f89-twvr2"]
Jul 28 12:57:45.263240 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.263222    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.270459 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.270435    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 12:57:45.271912 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.271889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 12:57:45.272396 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.272373    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 12:57:45.272888 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.272854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 12:57:45.276228 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.276210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4bhr\" (UniqueName: \"kubernetes.io/projected/ef38dd5b-65d2-4606-83cf-5931a7823eaf-kube-api-access-h4bhr\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.276302 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.276237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tmp\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.276339 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.276315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.283136 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.283118    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5jrsj\""
Jul 28 12:57:45.284060 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.284038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7897958f89-twvr2"]
Jul 28 12:57:45.376786 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.376731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tmp\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.376955 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.376833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.376955 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.376901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4bhr\" (UniqueName: \"kubernetes.io/projected/ef38dd5b-65d2-4606-83cf-5931a7823eaf-kube-api-access-h4bhr\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.377025 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:57:45.376989    2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 12:57:45.377067 ip-10-0-132-196 kubenswrapper[2562]: E0728 12:57:45.377056    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs podName:ef38dd5b-65d2-4606-83cf-5931a7823eaf nodeName:}" failed. No retries permitted until 2026-07-28 12:57:45.877037973 +0000 UTC m=+379.194056944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs") pod "kube-auth-proxy-7897958f89-twvr2" (UID: "ef38dd5b-65d2-4606-83cf-5931a7823eaf") : secret "kube-auth-proxy-tls" not found
Jul 28 12:57:45.379080 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.379054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tmp\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.398579 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.398550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4bhr\" (UniqueName: \"kubernetes.io/projected/ef38dd5b-65d2-4606-83cf-5931a7823eaf-kube-api-access-h4bhr\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.880899 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.880857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:45.883554 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:45.883526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef38dd5b-65d2-4606-83cf-5931a7823eaf-tls-certs\") pod \"kube-auth-proxy-7897958f89-twvr2\" (UID: \"ef38dd5b-65d2-4606-83cf-5931a7823eaf\") " pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:46.172462 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:46.172381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2"
Jul 28 12:57:46.320712 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:46.320688    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7897958f89-twvr2"]
Jul 28 12:57:46.322720 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:57:46.322695    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef38dd5b_65d2_4606_83cf_5931a7823eaf.slice/crio-c781ad896f02e5c03ef7878f51a04b38f97b9b9fa1fee5ac63cb6561d6452334 WatchSource:0}: Error finding container c781ad896f02e5c03ef7878f51a04b38f97b9b9fa1fee5ac63cb6561d6452334: Status 404 returned error can't find the container with id c781ad896f02e5c03ef7878f51a04b38f97b9b9fa1fee5ac63cb6561d6452334
Jul 28 12:57:47.047998 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:47.047956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2" event={"ID":"ef38dd5b-65d2-4606-83cf-5931a7823eaf","Type":"ContainerStarted","Data":"c781ad896f02e5c03ef7878f51a04b38f97b9b9fa1fee5ac63cb6561d6452334"}
Jul 28 12:57:49.060948 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:49.060910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2" event={"ID":"ef38dd5b-65d2-4606-83cf-5931a7823eaf","Type":"ContainerStarted","Data":"28bef61036c0a590763a09ab28316dda235b422ac3ea887ee6786cdc0e1b995d"}
Jul 28 12:57:49.080241 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:57:49.080187    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7897958f89-twvr2" podStartSLOduration=1.443300646 podStartE2EDuration="4.080167966s" totalImagesPullingTime="2.63686732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:57:49.078971742 +0000 UTC m=+382.395990751" watchObservedRunningTime="2026-07-28 12:57:49.080167966 +0000 UTC m=+382.397186956"
Jul 28 12:59:27.596842 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.596807    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"]
Jul 28 12:59:27.600254 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.600233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:27.608272 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.608251    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 12:59:27.608401 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.608275    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 12:59:27.609336 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.609314    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xm6cd\""
Jul 28 12:59:27.623838 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.623808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"]
Jul 28 12:59:27.715270 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.715236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h72wk\" (UniqueName: \"kubernetes.io/projected/8607ad9c-3b18-4197-a774-152e6be53fd1-kube-api-access-h72wk\") pod \"authorino-operator-75cc8d5cb7-g5crp\" (UID: \"8607ad9c-3b18-4197-a774-152e6be53fd1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:27.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.816417    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h72wk\" (UniqueName: \"kubernetes.io/projected/8607ad9c-3b18-4197-a774-152e6be53fd1-kube-api-access-h72wk\") pod \"authorino-operator-75cc8d5cb7-g5crp\" (UID: \"8607ad9c-3b18-4197-a774-152e6be53fd1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:27.836821 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.836796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h72wk\" (UniqueName: \"kubernetes.io/projected/8607ad9c-3b18-4197-a774-152e6be53fd1-kube-api-access-h72wk\") pod \"authorino-operator-75cc8d5cb7-g5crp\" (UID: \"8607ad9c-3b18-4197-a774-152e6be53fd1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:27.910398 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:27.910317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:28.046428 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:28.046401    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"]
Jul 28 12:59:28.048551 ip-10-0-132-196 kubenswrapper[2562]: W0728 12:59:28.048523    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8607ad9c_3b18_4197_a774_152e6be53fd1.slice/crio-67960a56d217690a0dae099797e7437a5e32b92efdbbe216ca1b7ae3b2b187a9 WatchSource:0}: Error finding container 67960a56d217690a0dae099797e7437a5e32b92efdbbe216ca1b7ae3b2b187a9: Status 404 returned error can't find the container with id 67960a56d217690a0dae099797e7437a5e32b92efdbbe216ca1b7ae3b2b187a9
Jul 28 12:59:28.506449 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:28.506414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp" event={"ID":"8607ad9c-3b18-4197-a774-152e6be53fd1","Type":"ContainerStarted","Data":"67960a56d217690a0dae099797e7437a5e32b92efdbbe216ca1b7ae3b2b187a9"}
Jul 28 12:59:30.517367 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:30.517328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp" event={"ID":"8607ad9c-3b18-4197-a774-152e6be53fd1","Type":"ContainerStarted","Data":"6821fe6b3e4eccfb97e94f50426a13160d5f3a5bc59f852f4b62642eb473da67"}
Jul 28 12:59:30.517789 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:30.517415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 12:59:30.561623 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:30.561531    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp" podStartSLOduration=1.329897818 podStartE2EDuration="3.561516538s" totalImagesPullingTime="2.23161872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:59:30.560040851 +0000 UTC m=+483.877059854" watchObservedRunningTime="2026-07-28 12:59:30.561516538 +0000 UTC m=+483.878535527"
Jul 28 12:59:41.524438 ip-10-0-132-196 kubenswrapper[2562]: I0728 12:59:41.524403    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g5crp"
Jul 28 13:00:25.762297 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.762214    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:00:25.765903 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.765881    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.772616 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.772307    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 13:00:25.772616 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.772340    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kwnnr\""
Jul 28 13:00:25.781082 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.781058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:00:25.832059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.832029    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:00:25.884664 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.884627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81f13ddb-42d0-4d33-8e13-52f17557f3e6-config-file\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.884874 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.884715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrgcp\" (UniqueName: \"kubernetes.io/projected/81f13ddb-42d0-4d33-8e13-52f17557f3e6-kube-api-access-rrgcp\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.986170 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.986132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrgcp\" (UniqueName: \"kubernetes.io/projected/81f13ddb-42d0-4d33-8e13-52f17557f3e6-kube-api-access-rrgcp\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.986353 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.986200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81f13ddb-42d0-4d33-8e13-52f17557f3e6-config-file\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.986819 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.986795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81f13ddb-42d0-4d33-8e13-52f17557f3e6-config-file\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:25.998179 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:25.998156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrgcp\" (UniqueName: \"kubernetes.io/projected/81f13ddb-42d0-4d33-8e13-52f17557f3e6-kube-api-access-rrgcp\") pod \"limitador-limitador-54d6d988d5-72hhp\" (UID: \"81f13ddb-42d0-4d33-8e13-52f17557f3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:26.075843 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:26.075714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:26.227383 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:26.227349    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:00:26.230649 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:00:26.230617    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f13ddb_42d0_4d33_8e13_52f17557f3e6.slice/crio-eef6ea987810ee02a1a3152f463962eeddc9a366c8e3b0a135446bcd486b88b7 WatchSource:0}: Error finding container eef6ea987810ee02a1a3152f463962eeddc9a366c8e3b0a135446bcd486b88b7: Status 404 returned error can't find the container with id eef6ea987810ee02a1a3152f463962eeddc9a366c8e3b0a135446bcd486b88b7
Jul 28 13:00:26.771996 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:26.771960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp" event={"ID":"81f13ddb-42d0-4d33-8e13-52f17557f3e6","Type":"ContainerStarted","Data":"eef6ea987810ee02a1a3152f463962eeddc9a366c8e3b0a135446bcd486b88b7"}
Jul 28 13:00:29.788029 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:29.787982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp" event={"ID":"81f13ddb-42d0-4d33-8e13-52f17557f3e6","Type":"ContainerStarted","Data":"c8419dc454b183d9471f5da903b813e3d28df47655dc011a9c5a8ac5a7abaeb6"}
Jul 28 13:00:29.788477 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:29.788069    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:00:29.809267 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:29.809219    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp" podStartSLOduration=1.8793052449999998 podStartE2EDuration="4.809205167s" totalImagesPullingTime="2.929899922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:00:29.808144134 +0000 UTC m=+543.125163138" watchObservedRunningTime="2026-07-28 13:00:29.809205167 +0000 UTC m=+543.126224156"
Jul 28 13:00:40.793901 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:00:40.793862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-72hhp"
Jul 28 13:01:46.903040 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:01:46.903004    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:02:15.411341 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:02:15.411304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:02:35.192385 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:02:35.192352    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:02:46.094329 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:02:46.094290    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:02:51.200076 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:02:51.200033    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:03:02.193279 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:02.193242    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:03:11.386048 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.386005    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"]
Jul 28 13:03:11.389727 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.389673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.393722 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.393692    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 13:03:11.393861 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.393724    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 13:03:11.393861 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.393703    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-llgkj\""
Jul 28 13:03:11.393861 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.393762    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 13:03:11.402025 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.402003    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"]
Jul 28 13:03:11.481493 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rbw\" (UniqueName: \"kubernetes.io/projected/8006dbcb-9735-4495-abea-75bcfbbe43cc-kube-api-access-d2rbw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.481657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.481657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8006dbcb-9735-4495-abea-75bcfbbe43cc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.481657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.481657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.481824 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.481724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582536 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8006dbcb-9735-4495-abea-75bcfbbe43cc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582716 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582716 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582716 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582716 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rbw\" (UniqueName: \"kubernetes.io/projected/8006dbcb-9735-4495-abea-75bcfbbe43cc-kube-api-access-d2rbw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.582716 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.582659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.583034 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.583008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.583107 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.583035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.583107 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.583084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.585097 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.585080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8006dbcb-9735-4495-abea-75bcfbbe43cc-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.585268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.585250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8006dbcb-9735-4495-abea-75bcfbbe43cc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.592604 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.592579    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rbw\" (UniqueName: \"kubernetes.io/projected/8006dbcb-9735-4495-abea-75bcfbbe43cc-kube-api-access-d2rbw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf\" (UID: \"8006dbcb-9735-4495-abea-75bcfbbe43cc\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.698986 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.698905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:11.831171 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.831139    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"]
Jul 28 13:03:11.833864 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:03:11.833830    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8006dbcb_9735_4495_abea_75bcfbbe43cc.slice/crio-84ad5d7bda867776a5dce2f3647c75fa3c9f1c0eecd2d416fa5e67febc9735c6 WatchSource:0}: Error finding container 84ad5d7bda867776a5dce2f3647c75fa3c9f1c0eecd2d416fa5e67febc9735c6: Status 404 returned error can't find the container with id 84ad5d7bda867776a5dce2f3647c75fa3c9f1c0eecd2d416fa5e67febc9735c6
Jul 28 13:03:11.835271 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:11.835254    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:03:12.529622 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:12.529578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf" event={"ID":"8006dbcb-9735-4495-abea-75bcfbbe43cc","Type":"ContainerStarted","Data":"84ad5d7bda867776a5dce2f3647c75fa3c9f1c0eecd2d416fa5e67febc9735c6"}
Jul 28 13:03:13.535752 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:13.535700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf" event={"ID":"8006dbcb-9735-4495-abea-75bcfbbe43cc","Type":"ContainerStarted","Data":"7633a67a52b92720030ff4ed59640b9bf004b06978a95b7f9d47366a81fd3add"}
Jul 28 13:03:13.555210 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:13.555161    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf" podStartSLOduration=1.51587216 podStartE2EDuration="2.555146717s" totalImagesPullingTime="1.039274557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:03:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:03:13.553008225 +0000 UTC m=+706.870027217" watchObservedRunningTime="2026-07-28 13:03:13.555146717 +0000 UTC m=+706.872165707"
Jul 28 13:03:14.401981 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:14.401941    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:03:21.699409 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:21.699319    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:21.699409 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:21.699370    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:21.711859 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:21.711836    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:03:22.589067 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:03:22.589042    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf"
Jul 28 13:04:23.396802 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:04:23.396759    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:04:33.001221 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:04:33.001183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:04:42.591391 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:04:42.591359    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:06:27.305130 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:06:27.305086    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:06:36.502338 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:06:36.502304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:05.797078 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:05.797038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:15.600693 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:15.600654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:25.587575 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:25.587538    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:33.586666 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:33.586626    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:43.214275 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:43.214230    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:07:53.304459 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:07:53.304374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:08:04.998100 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:08:04.998067    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:08:14.794245 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:08:14.794205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:08:37.982684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:08:37.982645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:08:47.398820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:08:47.398786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:09:11.397668 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:09:11.397630    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:09:22.097890 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:09:22.097811    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:09:33.095685 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:09:33.095644    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:09:50.401729 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:09:50.401693    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:10:01.198068 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:10:01.198024    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:10:11.591401 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:10:11.591357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:10:23.887609 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:10:23.887565    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:10:34.696507 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:10:34.696460    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:10:44.794001 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:10:44.793969    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:11:06.891875 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:11:06.891781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:12:35.394927 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:12:35.394889    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:12:41.888914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:12:41.888878    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:12:55.196686 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:12:55.196653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:13:07.005392 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:13:07.005356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:13:16.995494 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:13:16.995461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:13:27.889401 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:13:27.889366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:13:38.292508 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:13:38.292475    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:13:48.289392 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:13:48.289309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:00.690348 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:00.690312    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:09.492131 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:09.492099    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:22.097781 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:22.097728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:43.303073 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:43.303033    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:53.100496 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:53.100458    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:14:59.396292 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.396252    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:14:59.399799 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.399782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.402316 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.402290    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cdee320c-kserve-self-signed-certs\""
Jul 28 13:14:59.425546 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.425502    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:14:59.486551 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486509    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.486776 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.486776 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.486776 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5whv\" (UniqueName: \"kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.486776 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486729    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.486957 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.486789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588173 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588173 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588416 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5whv\" (UniqueName: \"kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588416 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588416 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588416 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588646 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588696 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588638    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.588734 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.588697    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.590636 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.590619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.590828 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.590808    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.596703 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.596683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5whv\" (UniqueName: \"kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv\") pod \"e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") " pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.710176 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.710092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:14:59.842155 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:14:59.842129    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:14:59.844550 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:14:59.844518    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaed5ecf6_c313_4060_970a_6ca6851eb2e3.slice/crio-7bb45d275c013a8f8a63f7ebcd4ecb516627cf16dc36245f07705ad82bcda1d5 WatchSource:0}: Error finding container 7bb45d275c013a8f8a63f7ebcd4ecb516627cf16dc36245f07705ad82bcda1d5: Status 404 returned error can't find the container with id 7bb45d275c013a8f8a63f7ebcd4ecb516627cf16dc36245f07705ad82bcda1d5
Jul 28 13:15:00.732089 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:00.732046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" event={"ID":"aed5ecf6-c313-4060-970a-6ca6851eb2e3","Type":"ContainerStarted","Data":"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"}
Jul 28 13:15:00.732481 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:00.732092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" event={"ID":"aed5ecf6-c313-4060-970a-6ca6851eb2e3","Type":"ContainerStarted","Data":"7bb45d275c013a8f8a63f7ebcd4ecb516627cf16dc36245f07705ad82bcda1d5"}
Jul 28 13:15:00.753863 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:00.753798    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" podStartSLOduration=1.753778844 podStartE2EDuration="1.753778844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:15:00.75237221 +0000 UTC m=+1414.069391199" watchObservedRunningTime="2026-07-28 13:15:00.753778844 +0000 UTC m=+1414.070797836"
Jul 28 13:15:02.989634 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:02.989599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:15:05.088232 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.088197    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:05.092274 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.092254    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.094805 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.094786    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cdee320c-kserve-self-signed-certs\""
Jul 28 13:15:05.101495 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.101470    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:05.244725 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.244925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.244925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.244925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.244925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk2s6\" (UniqueName: \"kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.245057 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.244957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk2s6\" (UniqueName: \"kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346678 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346678 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346678 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346994 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.346994 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.346964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.349016 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.348994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.349016 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.349011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.354157 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.354134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk2s6\" (UniqueName: \"kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6\") pod \"e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") " pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.402668 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.402628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:05.543908 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.543880    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:05.546134 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:15:05.546105    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e2f88b_0abe_4fe2_8d1c_5593f630e680.slice/crio-9aedc2430dc5c16d7c81e8a475d35945fc31f44b9ea967fc017330597121899c WatchSource:0}: Error finding container 9aedc2430dc5c16d7c81e8a475d35945fc31f44b9ea967fc017330597121899c: Status 404 returned error can't find the container with id 9aedc2430dc5c16d7c81e8a475d35945fc31f44b9ea967fc017330597121899c
Jul 28 13:15:05.755405 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.755371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" event={"ID":"75e2f88b-0abe-4fe2-8d1c-5593f630e680","Type":"ContainerStarted","Data":"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"}
Jul 28 13:15:05.755405 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.755408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" event={"ID":"75e2f88b-0abe-4fe2-8d1c-5593f630e680","Type":"ContainerStarted","Data":"9aedc2430dc5c16d7c81e8a475d35945fc31f44b9ea967fc017330597121899c"}
Jul 28 13:15:05.777717 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:05.777669    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" podStartSLOduration=0.777652941 podStartE2EDuration="777.652941ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:15:05.775940284 +0000 UTC m=+1419.092959488" watchObservedRunningTime="2026-07-28 13:15:05.777652941 +0000 UTC m=+1419.094671931"
Jul 28 13:15:09.710557 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:09.710473    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:09.710952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:09.710729    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:09.722905 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:09.722874    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:09.784799 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:09.784772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:15.402851 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:15.402820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:15.403412 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:15.403058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:15.415506 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:15.415474    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:15.812679 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:15.812653    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:19.096733 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:19.096652    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:15:27.840227 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:27.840186    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:15:28.603354 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:28.603320    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:32.392629 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:32.392596    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:15:42.860012 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:42.859952    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" podUID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" containerName="main" containerID="cri-o://afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f" gracePeriod=45
Jul 28 13:15:43.110758 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.110673    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:43.206825 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.206785    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.206984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.206844    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.206984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.206907    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.206984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.206943    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.207091 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207016    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5whv\" (UniqueName: \"kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.207091 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir\") pod \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\" (UID: \"aed5ecf6-c313-4060-970a-6ca6851eb2e3\") "
Jul 28 13:15:43.207199 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207137    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:43.207199 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207186    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home" (OuterVolumeSpecName: "home") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:43.207405 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207353    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:43.207405 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207390    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.207576 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.207430    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.209264 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.209239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm" (OuterVolumeSpecName: "dshm") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:43.209264 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.209247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv" (OuterVolumeSpecName: "kube-api-access-n5whv") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "kube-api-access-n5whv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:15:43.209407 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.209264    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aed5ecf6-c313-4060-970a-6ca6851eb2e3" (UID: "aed5ecf6-c313-4060-970a-6ca6851eb2e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:15:43.307952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.307923    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.307952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.307950    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.307952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.307959    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5whv\" (UniqueName: \"kubernetes.io/projected/aed5ecf6-c313-4060-970a-6ca6851eb2e3-kube-api-access-n5whv\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.308172 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.307968    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed5ecf6-c313-4060-970a-6ca6851eb2e3-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:43.623094 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.623034    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" podUID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" containerName="main" containerID="cri-o://e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697" gracePeriod=45
Jul 28 13:15:43.859351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.859325    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:43.896916 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.896818    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:15:43.937005 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.936973    2562 generic.go:356] "Generic (PLEG): container finished" podID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" containerID="afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f" exitCode=0
Jul 28 13:15:43.937191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.937012    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"
Jul 28 13:15:43.937191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.937057    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" event={"ID":"aed5ecf6-c313-4060-970a-6ca6851eb2e3","Type":"ContainerDied","Data":"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"}
Jul 28 13:15:43.937191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.937099    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p" event={"ID":"aed5ecf6-c313-4060-970a-6ca6851eb2e3","Type":"ContainerDied","Data":"7bb45d275c013a8f8a63f7ebcd4ecb516627cf16dc36245f07705ad82bcda1d5"}
Jul 28 13:15:43.937191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.937123    2562 scope.go:117] "RemoveContainer" containerID="afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"
Jul 28 13:15:43.938623 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.938428    2562 generic.go:356] "Generic (PLEG): container finished" podID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" containerID="e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697" exitCode=0
Jul 28 13:15:43.938623 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.938474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" event={"ID":"75e2f88b-0abe-4fe2-8d1c-5593f630e680","Type":"ContainerDied","Data":"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"}
Jul 28 13:15:43.938623 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.938517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc" event={"ID":"75e2f88b-0abe-4fe2-8d1c-5593f630e680","Type":"ContainerDied","Data":"9aedc2430dc5c16d7c81e8a475d35945fc31f44b9ea967fc017330597121899c"}
Jul 28 13:15:43.938623 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.938482    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"
Jul 28 13:15:43.946032 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.946010    2562 scope.go:117] "RemoveContainer" containerID="afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"
Jul 28 13:15:43.946280 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:15:43.946258    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f\": container with ID starting with afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f not found: ID does not exist" containerID="afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"
Jul 28 13:15:43.946333 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.946290    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f"} err="failed to get container status \"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f\": rpc error: code = NotFound desc = could not find container \"afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f\": container with ID starting with afea1712b91f522c5a638d48b7edb027ab31a8adb1257c477fa4c177067a842f not found: ID does not exist"
Jul 28 13:15:43.946333 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.946307    2562 scope.go:117] "RemoveContainer" containerID="e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"
Jul 28 13:15:43.954348 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.954325    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:15:43.955790 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.955769    2562 scope.go:117] "RemoveContainer" containerID="e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"
Jul 28 13:15:43.956062 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:15:43.956045    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697\": container with ID starting with e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697 not found: ID does not exist" containerID="e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"
Jul 28 13:15:43.956134 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.956068    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697"} err="failed to get container status \"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697\": rpc error: code = NotFound desc = could not find container \"e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697\": container with ID starting with e9db1cb676176d0236a4cd0e284c296dd2f6e7aee4cc90cbe9b48a9c52264697 not found: ID does not exist"
Jul 28 13:15:43.956480 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:43.956462    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cdee320c-kserve-84d5686d79-xlc5p"]
Jul 28 13:15:44.013973 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.013945    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014126 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.013987    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014126 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk2s6\" (UniqueName: \"kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014126 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014077    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014126 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014408 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014147    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home\") pod \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\" (UID: \"75e2f88b-0abe-4fe2-8d1c-5593f630e680\") "
Jul 28 13:15:44.014408 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014298    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:44.014520 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014396    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache" (OuterVolumeSpecName: "model-cache") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:44.014572 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.014563    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.015035 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.015008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home" (OuterVolumeSpecName: "home") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:44.016293 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.016262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:15:44.016378 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.016337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm" (OuterVolumeSpecName: "dshm") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:15:44.016509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.016490    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6" (OuterVolumeSpecName: "kube-api-access-vk2s6") pod "75e2f88b-0abe-4fe2-8d1c-5593f630e680" (UID: "75e2f88b-0abe-4fe2-8d1c-5593f630e680"). InnerVolumeSpecName "kube-api-access-vk2s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:15:44.115684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.115650    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk2s6\" (UniqueName: \"kubernetes.io/projected/75e2f88b-0abe-4fe2-8d1c-5593f630e680-kube-api-access-vk2s6\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.115684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.115677    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e2f88b-0abe-4fe2-8d1c-5593f630e680-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.115684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.115687    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.115684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.115694    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.115956 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.115701    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e2f88b-0abe-4fe2-8d1c-5593f630e680-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:15:44.259949 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.259917    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:44.263897 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:44.263875    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cdee320c-kserve-64cb5599c5-swdxc"]
Jul 28 13:15:45.252506 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:45.252468    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" path="/var/lib/kubelet/pods/75e2f88b-0abe-4fe2-8d1c-5593f630e680/volumes"
Jul 28 13:15:45.252953 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:45.252937    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" path="/var/lib/kubelet/pods/aed5ecf6-c313-4060-970a-6ca6851eb2e3/volumes"
Jul 28 13:15:50.701259 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:15:50.701228    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:06.802066 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:06.802020    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:17.050043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:17.050009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:26.404083 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:26.404046    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:34.294536 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:34.294500    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:44.303843 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:44.303807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:16:54.102353 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:16:54.102272    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:17:07.888116 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:17:07.888076    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:17:17.889820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:17:17.889783    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:17:27.100755 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:17:27.100698    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:17:39.800184 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:17:39.800145    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:17:51.788468 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:17:51.788435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:00.694094 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:00.694059    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:13.204732 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:13.204688    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:25.290648 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:25.290570    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:38.486226 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:38.486177    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:47.993400 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:47.993365    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:18:59.589503 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:18:59.589459    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:19:08.991579 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:08.991541    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:19:31.297153 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:31.297111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:19:49.285954 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.285874    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"]
Jul 28 13:19:49.286308 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286217    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" containerName="main"
Jul 28 13:19:49.286308 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286227    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" containerName="main"
Jul 28 13:19:49.286308 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286237    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" containerName="main"
Jul 28 13:19:49.286308 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286242    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" containerName="main"
Jul 28 13:19:49.286431 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286312    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed5ecf6-c313-4060-970a-6ca6851eb2e3" containerName="main"
Jul 28 13:19:49.286431 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.286322    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e2f88b-0abe-4fe2-8d1c-5593f630e680" containerName="main"
Jul 28 13:19:49.289150 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.289133    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:19:49.291517 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.291498    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:19:49.291632 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.291553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:19:49.297836 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.297813    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"]
Jul 28 13:19:49.390359 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.390323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx\" (UID: \"97205974-2b5c-456b-b8ca-ae41eb635cb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:19:49.491813 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.491776    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx\" (UID: \"97205974-2b5c-456b-b8ca-ae41eb635cb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:19:49.492403 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.492383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx\" (UID: \"97205974-2b5c-456b-b8ca-ae41eb635cb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:19:49.601151 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.601072    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:19:49.733487 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.733457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"]
Jul 28 13:19:49.735730 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:19:49.735705    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97205974_2b5c_456b_b8ca_ae41eb635cb6.slice/crio-b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84 WatchSource:0}: Error finding container b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84: Status 404 returned error can't find the container with id b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84
Jul 28 13:19:49.737773 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:49.737729    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:19:50.047227 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:19:50.047183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx" event={"ID":"97205974-2b5c-456b-b8ca-ae41eb635cb6","Type":"ContainerStarted","Data":"b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84"}
Jul 28 13:20:02.105653 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:02.105558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx" event={"ID":"97205974-2b5c-456b-b8ca-ae41eb635cb6","Type":"ContainerStarted","Data":"739f2b759efc81f56a57b065058b2634f648d85a3ecea211533230aa02be2ef5"}
Jul 28 13:20:02.121671 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:02.121622    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx" podStartSLOduration=1.285589783 podStartE2EDuration="13.121608439s" totalImagesPullingTime="11.836018656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:20:02.121229679 +0000 UTC m=+1715.438248670" watchObservedRunningTime="2026-07-28 13:20:02.121608439 +0000 UTC m=+1715.438627407"
Jul 28 13:20:03.112263 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:03.112227    2562 generic.go:356] "Generic (PLEG): container finished" podID="97205974-2b5c-456b-b8ca-ae41eb635cb6" containerID="739f2b759efc81f56a57b065058b2634f648d85a3ecea211533230aa02be2ef5" exitCode=0
Jul 28 13:20:03.112645 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:03.112316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx" event={"ID":"97205974-2b5c-456b-b8ca-ae41eb635cb6","Type":"ContainerDied","Data":"739f2b759efc81f56a57b065058b2634f648d85a3ecea211533230aa02be2ef5"}
Jul 28 13:20:04.243374 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:04.243349    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:20:04.321510 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:04.321467    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca\") pod \"97205974-2b5c-456b-b8ca-ae41eb635cb6\" (UID: \"97205974-2b5c-456b-b8ca-ae41eb635cb6\") "
Jul 28 13:20:04.321880 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:04.321854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97205974-2b5c-456b-b8ca-ae41eb635cb6" (UID: "97205974-2b5c-456b-b8ca-ae41eb635cb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:20:04.422979 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:04.422904    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97205974-2b5c-456b-b8ca-ae41eb635cb6-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:20:05.122684 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:05.122646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx" event={"ID":"97205974-2b5c-456b-b8ca-ae41eb635cb6","Type":"ContainerDied","Data":"b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84"}
Jul 28 13:20:05.122869 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:05.122697    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6b4829338c0a8f6433617a0f53887a45250c6771eda1a10983a8bdc71116e84"
Jul 28 13:20:05.122869 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:05.122663    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"
Jul 28 13:20:14.576540 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:14.576504    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"]
Jul 28 13:20:14.580051 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:14.580026    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0142f26b-j4mhx"]
Jul 28 13:20:15.251802 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:15.251768    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97205974-2b5c-456b-b8ca-ae41eb635cb6" path="/var/lib/kubelet/pods/97205974-2b5c-456b-b8ca-ae41eb635cb6/volumes"
Jul 28 13:20:18.197322 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:18.197286    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:20:28.903938 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:28.903904    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:20:53.310469 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:20:53.310428    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:21:05.790273 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:05.790236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:21:17.976325 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.976246    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"]
Jul 28 13:21:17.976679 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.976667    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97205974-2b5c-456b-b8ca-ae41eb635cb6" containerName="revoke-keys"
Jul 28 13:21:17.976679 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.976679    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="97205974-2b5c-456b-b8ca-ae41eb635cb6" containerName="revoke-keys"
Jul 28 13:21:17.976768 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.976735    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="97205974-2b5c-456b-b8ca-ae41eb635cb6" containerName="revoke-keys"
Jul 28 13:21:17.979784 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.979764    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:17.982145 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.982116    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:21:17.982273 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.982233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:21:17.986667 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:17.986645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"]
Jul 28 13:21:18.087492 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.087456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5\" (UID: \"48a80577-057e-4eed-8a16-1b624b58183a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:18.188849 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.188809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5\" (UID: \"48a80577-057e-4eed-8a16-1b624b58183a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:18.189451 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.189429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5\" (UID: \"48a80577-057e-4eed-8a16-1b624b58183a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:18.291674 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.291634    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:18.425127 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.425002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"]
Jul 28 13:21:18.427427 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:21:18.427402    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a80577_057e_4eed_8a16_1b624b58183a.slice/crio-664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70 WatchSource:0}: Error finding container 664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70: Status 404 returned error can't find the container with id 664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70
Jul 28 13:21:18.455173 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:18.455146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5" event={"ID":"48a80577-057e-4eed-8a16-1b624b58183a","Type":"ContainerStarted","Data":"664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70"}
Jul 28 13:21:19.466783 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:19.466678    2562 generic.go:356] "Generic (PLEG): container finished" podID="48a80577-057e-4eed-8a16-1b624b58183a" containerID="0b9614ffbc01d178dc33f97079790d7e835652efa8b4cbc7511e776a014f2342" exitCode=0
Jul 28 13:21:19.466783 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:19.466761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5" event={"ID":"48a80577-057e-4eed-8a16-1b624b58183a","Type":"ContainerDied","Data":"0b9614ffbc01d178dc33f97079790d7e835652efa8b4cbc7511e776a014f2342"}
Jul 28 13:21:20.599856 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:20.599831    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:20.714006 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:20.713975    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca\") pod \"48a80577-057e-4eed-8a16-1b624b58183a\" (UID: \"48a80577-057e-4eed-8a16-1b624b58183a\") "
Jul 28 13:21:20.714271 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:20.714250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48a80577-057e-4eed-8a16-1b624b58183a" (UID: "48a80577-057e-4eed-8a16-1b624b58183a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:21:20.815158 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:20.815121    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a80577-057e-4eed-8a16-1b624b58183a-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:21:21.477534 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:21.477496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5" event={"ID":"48a80577-057e-4eed-8a16-1b624b58183a","Type":"ContainerDied","Data":"664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70"}
Jul 28 13:21:21.477700 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:21.477552    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="664153ba893f1fc69e80f987463356b661c7bc5555543dbb1378862ae22e1e70"
Jul 28 13:21:21.477700 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:21.477510    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"
Jul 28 13:21:23.056101 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:23.056061    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"]
Jul 28 13:21:23.058662 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:23.058632    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-72ce9b01-mz8k5"]
Jul 28 13:21:23.252325 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:23.252287    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a80577-057e-4eed-8a16-1b624b58183a" path="/var/lib/kubelet/pods/48a80577-057e-4eed-8a16-1b624b58183a/volumes"
Jul 28 13:21:27.309372 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:27.309334    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:21:39.485145 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:21:39.485107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:22:01.703578 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:22:01.703542    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:23:24.300705 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:23:24.300672    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:23:34.493758 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:23:34.493712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:23:46.096132 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:23:46.096052    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:23:58.993955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:23:58.993918    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:24:19.522831 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.522731    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"]
Jul 28 13:24:19.523189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.523158    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a80577-057e-4eed-8a16-1b624b58183a" containerName="revoke-keys"
Jul 28 13:24:19.523189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.523171    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a80577-057e-4eed-8a16-1b624b58183a" containerName="revoke-keys"
Jul 28 13:24:19.523259 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.523237    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a80577-057e-4eed-8a16-1b624b58183a" containerName="revoke-keys"
Jul 28 13:24:19.526275 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.526257    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:19.528962 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.528938    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:24:19.529077 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.529002    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:24:19.533186 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.533157    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"]
Jul 28 13:24:19.708768 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.708707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a757517a-q7mcq\" (UID: \"287dd001-1b64-46c2-b8dc-5fdef7ad9083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:19.809859 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.809726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a757517a-q7mcq\" (UID: \"287dd001-1b64-46c2-b8dc-5fdef7ad9083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:19.810348 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.810327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a757517a-q7mcq\" (UID: \"287dd001-1b64-46c2-b8dc-5fdef7ad9083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:19.838718 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.838686    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:19.973238 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:19.973213    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"]
Jul 28 13:24:19.975800 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:24:19.975774    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287dd001_1b64_46c2_b8dc_5fdef7ad9083.slice/crio-270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b WatchSource:0}: Error finding container 270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b: Status 404 returned error can't find the container with id 270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b
Jul 28 13:24:20.315037 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:20.315002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq" event={"ID":"287dd001-1b64-46c2-b8dc-5fdef7ad9083","Type":"ContainerStarted","Data":"270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b"}
Jul 28 13:24:21.320948 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:21.320912    2562 generic.go:356] "Generic (PLEG): container finished" podID="287dd001-1b64-46c2-b8dc-5fdef7ad9083" containerID="275511ca8f8e5fb9f562cdcda498a4a47049dd127c29d880a758e92e3b227779" exitCode=0
Jul 28 13:24:21.321340 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:21.321003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq" event={"ID":"287dd001-1b64-46c2-b8dc-5fdef7ad9083","Type":"ContainerDied","Data":"275511ca8f8e5fb9f562cdcda498a4a47049dd127c29d880a758e92e3b227779"}
Jul 28 13:24:22.463535 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:22.463509    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:22.531389 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:22.531358    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca\") pod \"287dd001-1b64-46c2-b8dc-5fdef7ad9083\" (UID: \"287dd001-1b64-46c2-b8dc-5fdef7ad9083\") "
Jul 28 13:24:22.531709 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:22.531687    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "287dd001-1b64-46c2-b8dc-5fdef7ad9083" (UID: "287dd001-1b64-46c2-b8dc-5fdef7ad9083"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:24:22.632809 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:22.632714    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/287dd001-1b64-46c2-b8dc-5fdef7ad9083-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:24:23.331730 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:23.331698    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"
Jul 28 13:24:23.331730 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:23.331705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq" event={"ID":"287dd001-1b64-46c2-b8dc-5fdef7ad9083","Type":"ContainerDied","Data":"270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b"}
Jul 28 13:24:23.331960 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:23.331776    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="270d91c382827f706086ca6b5172fb6e6c5098e9011089fba4594f94d5325e0b"
Jul 28 13:24:30.142243 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:30.142209    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"]
Jul 28 13:24:30.145263 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:30.145233    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a757517a-q7mcq"]
Jul 28 13:24:31.251464 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:31.251432    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287dd001-1b64-46c2-b8dc-5fdef7ad9083" path="/var/lib/kubelet/pods/287dd001-1b64-46c2-b8dc-5fdef7ad9083/volumes"
Jul 28 13:24:33.192528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:24:33.192492    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:25:05.586063 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:05.586030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:25:40.793125 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.793090    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:25:40.793728 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.793634    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287dd001-1b64-46c2-b8dc-5fdef7ad9083" containerName="revoke-keys"
Jul 28 13:25:40.793728 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.793654    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="287dd001-1b64-46c2-b8dc-5fdef7ad9083" containerName="revoke-keys"
Jul 28 13:25:40.793890 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.793809    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="287dd001-1b64-46c2-b8dc-5fdef7ad9083" containerName="revoke-keys"
Jul 28 13:25:40.796856 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.796836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.800196 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.800172    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11cb60ac\"/\"kube-root-ca.crt\""
Jul 28 13:25:40.800295 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.800193    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11cb60ac\"/\"openshift-service-ca.crt\""
Jul 28 13:25:40.800295 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.800213    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11cb60ac\"/\"e2e-lifecycle-model-11cb60ac-kserve-self-signed-certs\""
Jul 28 13:25:40.800295 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.800223    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11cb60ac\"/\"default-dockercfg-7vlxx\""
Jul 28 13:25:40.803781 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.803735    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:25:40.953042 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.953225 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.953225 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpzg4\" (UniqueName: \"kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.953225 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.953337 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:40.953337 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:40.953262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.053952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.053867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.053952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.053934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpzg4\" (UniqueName: \"kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054168 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.053986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054168 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054168 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054168 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054053    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054412 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054355    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.054412 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.054368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.056378 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.056356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.056683 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.056666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.061881 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.061855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpzg4\" (UniqueName: \"kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4\") pod \"e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") " pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.106914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.106865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:41.238482 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.238457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:25:41.241611 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:25:41.241579    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1738cf3_cdd2_475c_9ba7_2c3cae593b04.slice/crio-fc22fa67fe0d4463b0f758728a088e659b3ed1bf896a996e74a97787bc2719b6 WatchSource:0}: Error finding container fc22fa67fe0d4463b0f758728a088e659b3ed1bf896a996e74a97787bc2719b6: Status 404 returned error can't find the container with id fc22fa67fe0d4463b0f758728a088e659b3ed1bf896a996e74a97787bc2719b6
Jul 28 13:25:41.714825 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.714784    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" event={"ID":"d1738cf3-cdd2-475c-9ba7-2c3cae593b04","Type":"ContainerStarted","Data":"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"}
Jul 28 13:25:41.714998 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.714835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" event={"ID":"d1738cf3-cdd2-475c-9ba7-2c3cae593b04","Type":"ContainerStarted","Data":"fc22fa67fe0d4463b0f758728a088e659b3ed1bf896a996e74a97787bc2719b6"}
Jul 28 13:25:41.735056 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:41.735005    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" podStartSLOduration=1.73498826 podStartE2EDuration="1.73498826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:25:41.73394429 +0000 UTC m=+2055.050963310" watchObservedRunningTime="2026-07-28 13:25:41.73498826 +0000 UTC m=+2055.052007250"
Jul 28 13:25:44.192482 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:44.192442    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:25:51.107681 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:51.107580    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:51.107681 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:51.107642    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:51.120254 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:51.120228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:25:51.773641 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:25:51.773613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:26:01.089928 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:01.089889    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:26:05.557026 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.556991    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"]
Jul 28 13:26:05.560497 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.560478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:05.562879 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.562857    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:26:05.562879 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.562872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:26:05.566860 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.566786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"]
Jul 28 13:26:05.679200 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.679165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc\" (UID: \"a33734ea-e3e1-4f76-a083-4ecc18e91d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:05.779941 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.779905    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc\" (UID: \"a33734ea-e3e1-4f76-a083-4ecc18e91d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:05.780527 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.780505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc\" (UID: \"a33734ea-e3e1-4f76-a083-4ecc18e91d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:05.873345 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:05.873244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:06.006495 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:06.006470    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"]
Jul 28 13:26:06.008789 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:26:06.008729    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33734ea_e3e1_4f76_a083_4ecc18e91d3c.slice/crio-86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7 WatchSource:0}: Error finding container 86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7: Status 404 returned error can't find the container with id 86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7
Jul 28 13:26:06.834122 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:06.834091    2562 generic.go:356] "Generic (PLEG): container finished" podID="a33734ea-e3e1-4f76-a083-4ecc18e91d3c" containerID="1d0d438b869f5b594126212ba3af18a193726450ee37941b4b3a21a3be58f896" exitCode=0
Jul 28 13:26:06.834517 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:06.834183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc" event={"ID":"a33734ea-e3e1-4f76-a083-4ecc18e91d3c","Type":"ContainerDied","Data":"1d0d438b869f5b594126212ba3af18a193726450ee37941b4b3a21a3be58f896"}
Jul 28 13:26:06.834517 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:06.834232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc" event={"ID":"a33734ea-e3e1-4f76-a083-4ecc18e91d3c","Type":"ContainerStarted","Data":"86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7"}
Jul 28 13:26:07.972250 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:07.972223    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:08.102386 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.102304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca\") pod \"a33734ea-e3e1-4f76-a083-4ecc18e91d3c\" (UID: \"a33734ea-e3e1-4f76-a083-4ecc18e91d3c\") "
Jul 28 13:26:08.102663 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.102638    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a33734ea-e3e1-4f76-a083-4ecc18e91d3c" (UID: "a33734ea-e3e1-4f76-a083-4ecc18e91d3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:26:08.203450 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.203404    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a33734ea-e3e1-4f76-a083-4ecc18e91d3c-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.845005 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.844974    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc" event={"ID":"a33734ea-e3e1-4f76-a083-4ecc18e91d3c","Type":"ContainerDied","Data":"86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7"}
Jul 28 13:26:08.845192 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.845014    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"
Jul 28 13:26:08.845192 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:08.845022    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86222a34d6fbf86126220efcf410c2684cafb7aa5743b780120d9bb4f1963dc7"
Jul 28 13:26:14.968865 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:14.968829    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"]
Jul 28 13:26:14.971812 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:14.971787    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-11cb60ac-hv6hc"]
Jul 28 13:26:15.252343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:15.252304    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a33734ea-e3e1-4f76-a083-4ecc18e91d3c" path="/var/lib/kubelet/pods/a33734ea-e3e1-4f76-a083-4ecc18e91d3c/volumes"
Jul 28 13:26:17.696553 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:17.696524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:26:23.708941 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:23.708905    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:26:27.285952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:27.285921    2562 scope.go:117] "RemoveContainer" containerID="739f2b759efc81f56a57b065058b2634f648d85a3ecea211533230aa02be2ef5"
Jul 28 13:26:38.728896 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.728846    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" podUID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" containerName="main" containerID="cri-o://2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002" gracePeriod=45
Jul 28 13:26:38.986825 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.986756    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:26:38.989356 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.989326    2562 generic.go:356] "Generic (PLEG): container finished" podID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" containerID="2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002" exitCode=0
Jul 28 13:26:38.989473 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.989358    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"
Jul 28 13:26:38.989473 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.989405    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" event={"ID":"d1738cf3-cdd2-475c-9ba7-2c3cae593b04","Type":"ContainerDied","Data":"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"}
Jul 28 13:26:38.989473 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.989451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6" event={"ID":"d1738cf3-cdd2-475c-9ba7-2c3cae593b04","Type":"ContainerDied","Data":"fc22fa67fe0d4463b0f758728a088e659b3ed1bf896a996e74a97787bc2719b6"}
Jul 28 13:26:38.989473 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.989472    2562 scope.go:117] "RemoveContainer" containerID="2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"
Jul 28 13:26:38.997866 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.997847    2562 scope.go:117] "RemoveContainer" containerID="2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"
Jul 28 13:26:38.998216 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:26:38.998194    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002\": container with ID starting with 2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002 not found: ID does not exist" containerID="2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"
Jul 28 13:26:38.998273 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:38.998226    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002"} err="failed to get container status \"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002\": rpc error: code = NotFound desc = could not find container \"2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002\": container with ID starting with 2aa1ca1c35ae014a780966ec055c62fc3c66a8dd82b03991b741f5a95d994002 not found: ID does not exist"
Jul 28 13:26:39.078015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.077978    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078209 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078209 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078069    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078209 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078113    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078209 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpzg4\" (UniqueName: \"kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078209 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078190    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home\") pod \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\" (UID: \"d1738cf3-cdd2-475c-9ba7-2c3cae593b04\") "
Jul 28 13:26:39.078442 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078368    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:39.078442 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078373    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:39.078525 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078494    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home" (OuterVolumeSpecName: "home") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:39.078525 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078511    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.078602 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.078531    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.080406 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.080380    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm" (OuterVolumeSpecName: "dshm") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:39.080479 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.080436    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4" (OuterVolumeSpecName: "kube-api-access-fpzg4") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "kube-api-access-fpzg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:26:39.080526 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.080485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1738cf3-cdd2-475c-9ba7-2c3cae593b04" (UID: "d1738cf3-cdd2-475c-9ba7-2c3cae593b04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:26:39.179606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.179553    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.179606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.179596    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.179606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.179608    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpzg4\" (UniqueName: \"kubernetes.io/projected/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-kube-api-access-fpzg4\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.179606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.179617    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1738cf3-cdd2-475c-9ba7-2c3cae593b04-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:26:39.305464 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.305429    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:26:39.309083 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:39.309058    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-11cb60ac/e2e-lifecycle-model-11cb60ac-kserve-5b6b75fd47-khsw6"]
Jul 28 13:26:41.252115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:26:41.252074    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" path="/var/lib/kubelet/pods/d1738cf3-cdd2-475c-9ba7-2c3cae593b04/volumes"
Jul 28 13:27:11.719548 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:27:11.719502    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:27:21.903259 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:27:21.903176    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:27:27.304366 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:27:27.304327    2562 scope.go:117] "RemoveContainer" containerID="0b9614ffbc01d178dc33f97079790d7e835652efa8b4cbc7511e776a014f2342"
Jul 28 13:28:16.888463 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:16.888424    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:28:35.029764 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.029712    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030697    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33734ea-e3e1-4f76-a083-4ecc18e91d3c" containerName="revoke-keys"
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030721    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33734ea-e3e1-4f76-a083-4ecc18e91d3c" containerName="revoke-keys"
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030759    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" containerName="main"
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030781    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" containerName="main"
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030874    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1738cf3-cdd2-475c-9ba7-2c3cae593b04" containerName="main"
Jul 28 13:28:35.032208 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.030892    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33734ea-e3e1-4f76-a083-4ecc18e91d3c" containerName="revoke-keys"
Jul 28 13:28:35.034022 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.034003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.037630 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.037588    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d82b76-openshift-default-dockercfg-vh7jg\""
Jul 28 13:28:35.037778 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.037688    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 13:28:35.045089 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.045009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:28:35.143099 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143278 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143278 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143278 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143278 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpzl4\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.143420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.143385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244538 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpzl4\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.244762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245080 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245080 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245080 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.244999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245211 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.245072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245211 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.245096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245459 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.245378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.245659 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.245588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.249242 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.249219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.249443 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.249414    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.255022 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.255001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.255145 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.255129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpzl4\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4\") pod \"e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") " pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.348300 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.348213    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:35.490929 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.490901    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:28:35.493041 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:28:35.493009    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc20be5_f178_46ee_b372_a566d6028dbc.slice/crio-4385bd2192dd4d5567a4f4be414ba763a8a8d0658239cc2e9847e8710daa835d WatchSource:0}: Error finding container 4385bd2192dd4d5567a4f4be414ba763a8a8d0658239cc2e9847e8710daa835d: Status 404 returned error can't find the container with id 4385bd2192dd4d5567a4f4be414ba763a8a8d0658239cc2e9847e8710daa835d
Jul 28 13:28:35.495128 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.495040    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:28:35.540444 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:35.540413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerStarted","Data":"4385bd2192dd4d5567a4f4be414ba763a8a8d0658239cc2e9847e8710daa835d"}
Jul 28 13:28:38.359420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:38.359380    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:38.359669 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:38.359456    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:38.359669 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:38.359494    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:38.559834 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:38.559782    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerStarted","Data":"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"}
Jul 28 13:28:38.580792 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:38.580709    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podStartSLOduration=0.716765464 podStartE2EDuration="3.580662244s" totalImagesPullingTime="2.86389678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:28:38.580349775 +0000 UTC m=+2231.897368765" watchObservedRunningTime="2026-07-28 13:28:38.580662244 +0000 UTC m=+2231.897681238"
Jul 28 13:28:39.348817 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:39.348786    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:39.353864 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:39.353839    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:39.564913 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:39.564884    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:39.565850 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:39.565826    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:45.897961 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:45.897926    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:28:54.566102 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.566060    2562 patch_prober.go:28] interesting pod/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 13:28:54.566532 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.566146    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 13:28:54.648449 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.648418    2562 generic.go:356] "Generic (PLEG): container finished" podID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerID="c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02" exitCode=0
Jul 28 13:28:54.648628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.648492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerDied","Data":"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"}
Jul 28 13:28:54.648933 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.648919    2562 scope.go:117] "RemoveContainer" containerID="c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"
Jul 28 13:28:54.650992 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.650960    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:54.651079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.651030    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:54.651079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:54.651062    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:28:55.655154 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:55.655118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerStarted","Data":"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"}
Jul 28 13:28:56.349042 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:56.349003    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:28:56.349263 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:56.349240    2562 patch_prober.go:28] interesting pod/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 13:28:56.349348 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:56.349299    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 13:28:57.348574 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:57.348536    2562 patch_prober.go:28] interesting pod/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 13:28:57.349011 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:57.348592    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 13:28:58.348625 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:58.348588    2562 patch_prober.go:28] interesting pod/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 13:28:58.349068 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:58.348642    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 13:28:59.349546 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:59.349507    2562 patch_prober.go:28] interesting pod/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 13:28:59.349977 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:28:59.349573    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 13:29:00.353215 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:00.353185    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:29:00.353610 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:00.353464    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:29:00.354241 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:00.354225    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:29:15.697508 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.697462    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:29:15.700748 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.700721    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.704715 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.704684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8b4f72\"/\"default-dockercfg-npn6k\""
Jul 28 13:29:15.704918 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.704756    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8b4f72\"/\"openshift-service-ca.crt\""
Jul 28 13:29:15.704918 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.704785    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8b4f72\"/\"test-model-8b4f72-kserve-self-signed-certs\""
Jul 28 13:29:15.705045 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.704993    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8b4f72\"/\"kube-root-ca.crt\""
Jul 28 13:29:15.710964 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.710940    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:29:15.816268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.816453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.816426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2h4k\" (UniqueName: \"kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917206 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917206 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917452 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917452 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917452 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2h4k\" (UniqueName: \"kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917452 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917660 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917770 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.917839 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.917822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.919585 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.919564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.919942 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.919924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:15.926167 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:15.926144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2h4k\" (UniqueName: \"kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k\") pod \"test-model-8b4f72-kserve-566d9b4699-fn4rb\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") " pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:16.011592 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:16.011543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:16.150265 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:16.150237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:29:16.152174 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:29:16.152148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e84a31b_c4c4_4ab8_97c1_61fb7ceb30a9.slice/crio-c3fb6771366bbc4dc1acba24017ee00055e39f988f6737acaf22dba9564535bf WatchSource:0}: Error finding container c3fb6771366bbc4dc1acba24017ee00055e39f988f6737acaf22dba9564535bf: Status 404 returned error can't find the container with id c3fb6771366bbc4dc1acba24017ee00055e39f988f6737acaf22dba9564535bf
Jul 28 13:29:16.772093 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:16.772049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" event={"ID":"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9","Type":"ContainerStarted","Data":"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"}
Jul 28 13:29:16.772093 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:16.772097    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" event={"ID":"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9","Type":"ContainerStarted","Data":"c3fb6771366bbc4dc1acba24017ee00055e39f988f6737acaf22dba9564535bf"}
Jul 28 13:29:16.791259 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:16.791202    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" podStartSLOduration=1.7911840300000001 podStartE2EDuration="1.79118403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:29:16.789259644 +0000 UTC m=+2270.106278636" watchObservedRunningTime="2026-07-28 13:29:16.79118403 +0000 UTC m=+2270.108203020"
Jul 28 13:29:19.996435 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:19.996399    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:29:26.012515 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:26.012472    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:26.013037 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:26.012537    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:26.025144 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:26.025118    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:26.829588 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:26.829552    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:29:49.090920 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.090886    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:29:49.094490 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.094464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.097780 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.097754    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d82b76\"/\"openshift-service-ca.crt\""
Jul 28 13:29:49.097915 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.097754    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d82b76\"/\"test-model-d82b76-kserve-self-signed-certs\""
Jul 28 13:29:49.097915 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.097755    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d82b76\"/\"kube-root-ca.crt\""
Jul 28 13:29:49.097915 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.097840    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d82b76\"/\"default-dockercfg-l7zmp\""
Jul 28 13:29:49.103254 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.102896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:29:49.125135 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.125284 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plhd8\" (UniqueName: \"kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.125284 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.125284 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.125385 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.125418 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.125389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.226787 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.226787 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227035 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227035 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plhd8\" (UniqueName: \"kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227035 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227035 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.226901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227255 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.227163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227255 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.227233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.227330 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.227288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.229247 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.229228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.229512 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.229495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.237071 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.237052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plhd8\" (UniqueName: \"kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8\") pod \"test-model-d82b76-kserve-7c55d8f867-d4rs2\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") " pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.405763 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.405659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:49.544664 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.544630    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:29:49.549555 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:29:49.549530    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8eca1682_71c7_4d99_a0ff_1be336d28b88.slice/crio-27ad23966989e70a6e089a134646a48525e2d597d74387fbce930bc7b2d04c17 WatchSource:0}: Error finding container 27ad23966989e70a6e089a134646a48525e2d597d74387fbce930bc7b2d04c17: Status 404 returned error can't find the container with id 27ad23966989e70a6e089a134646a48525e2d597d74387fbce930bc7b2d04c17
Jul 28 13:29:49.933417 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.933381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" event={"ID":"8eca1682-71c7-4d99-a0ff-1be336d28b88","Type":"ContainerStarted","Data":"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"}
Jul 28 13:29:49.933417 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.933422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" event={"ID":"8eca1682-71c7-4d99-a0ff-1be336d28b88","Type":"ContainerStarted","Data":"27ad23966989e70a6e089a134646a48525e2d597d74387fbce930bc7b2d04c17"}
Jul 28 13:29:49.949664 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:49.949597    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" podStartSLOduration=0.949578934 podStartE2EDuration="949.578934ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:29:49.949559301 +0000 UTC m=+2303.266578275" watchObservedRunningTime="2026-07-28 13:29:49.949578934 +0000 UTC m=+2303.266597922"
Jul 28 13:29:52.897431 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:52.897395    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:29:59.406064 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:59.406025    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:59.406600 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:59.406087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:59.418633 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:59.418603    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:29:59.993597 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:29:59.993568    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:30:08.188837 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:08.188796    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:30:27.343029 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:27.342938    2562 scope.go:117] "RemoveContainer" containerID="275511ca8f8e5fb9f562cdcda498a4a47049dd127c29d880a758e92e3b227779"
Jul 28 13:30:30.777279 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:30.777241    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:30:31.730654 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:31.730610    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:30:35.792921 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:35.792891    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:30:45.798814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:45.798727    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" podUID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" containerName="main" containerID="cri-o://80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09" gracePeriod=45
Jul 28 13:30:46.045306 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.045280    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:30:46.182183 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182098    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182183 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182156    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182373 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2h4k\" (UniqueName: \"kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182373 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182265    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182373 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182286    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182373 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182318    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home\") pod \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\" (UID: \"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9\") "
Jul 28 13:30:46.182571 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:46.182571 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:46.182715 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182687    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home" (OuterVolumeSpecName: "home") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:46.182801 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182701    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.182801 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.182774    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.184701 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.184675    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm" (OuterVolumeSpecName: "dshm") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:46.184701 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.184686    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:30:46.184900 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.184713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k" (OuterVolumeSpecName: "kube-api-access-s2h4k") pod "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" (UID: "6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9"). InnerVolumeSpecName "kube-api-access-s2h4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:30:46.210057 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.210022    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" containerID="80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09" exitCode=0
Jul 28 13:30:46.210200 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.210066    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"
Jul 28 13:30:46.210200 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.210110    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" event={"ID":"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9","Type":"ContainerDied","Data":"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"}
Jul 28 13:30:46.210200 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.210153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb" event={"ID":"6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9","Type":"ContainerDied","Data":"c3fb6771366bbc4dc1acba24017ee00055e39f988f6737acaf22dba9564535bf"}
Jul 28 13:30:46.210200 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.210170    2562 scope.go:117] "RemoveContainer" containerID="80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"
Jul 28 13:30:46.218911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.218891    2562 scope.go:117] "RemoveContainer" containerID="80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"
Jul 28 13:30:46.219182 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:30:46.219162    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09\": container with ID starting with 80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09 not found: ID does not exist" containerID="80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"
Jul 28 13:30:46.219252 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.219191    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09"} err="failed to get container status \"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09\": rpc error: code = NotFound desc = could not find container \"80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09\": container with ID starting with 80b66fb9ec1cab4cc6a9255e6a0062e67a6a0cb9704bcd469c9ab75ecc922b09 not found: ID does not exist"
Jul 28 13:30:46.231524 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.231499    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:30:46.234631 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.234604    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8b4f72/test-model-8b4f72-kserve-566d9b4699-fn4rb"]
Jul 28 13:30:46.283954 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.283921    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2h4k\" (UniqueName: \"kubernetes.io/projected/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-kube-api-access-s2h4k\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.283954 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.283949    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.283954 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.283960    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.284191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.283967    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:46.750646 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.750599    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" podUID="8eca1682-71c7-4d99-a0ff-1be336d28b88" containerName="main" containerID="cri-o://91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d" gracePeriod=45
Jul 28 13:30:46.995992 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:46.995963    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:30:47.092024 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.091938    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plhd8\" (UniqueName: \"kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092024 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.091982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092040    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092090    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092151    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092193    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm\") pod \"8eca1682-71c7-4d99-a0ff-1be336d28b88\" (UID: \"8eca1682-71c7-4d99-a0ff-1be336d28b88\") "
Jul 28 13:30:47.092431 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092374    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache" (OuterVolumeSpecName: "model-cache") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:47.092431 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092419    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:47.092535 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home" (OuterVolumeSpecName: "home") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:47.092590 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092541    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.092590 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092563    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.092590 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.092576    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.094425 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.094403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:30:47.094501 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.094403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm" (OuterVolumeSpecName: "dshm") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:30:47.094501 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.094435    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8" (OuterVolumeSpecName: "kube-api-access-plhd8") pod "8eca1682-71c7-4d99-a0ff-1be336d28b88" (UID: "8eca1682-71c7-4d99-a0ff-1be336d28b88"). InnerVolumeSpecName "kube-api-access-plhd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:30:47.193688 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.193649    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eca1682-71c7-4d99-a0ff-1be336d28b88-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.193688 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.193682    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plhd8\" (UniqueName: \"kubernetes.io/projected/8eca1682-71c7-4d99-a0ff-1be336d28b88-kube-api-access-plhd8\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.193688 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.193693    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eca1682-71c7-4d99-a0ff-1be336d28b88-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:47.216078 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.216043    2562 generic.go:356] "Generic (PLEG): container finished" podID="8eca1682-71c7-4d99-a0ff-1be336d28b88" containerID="91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d" exitCode=0
Jul 28 13:30:47.216237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.216092    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"
Jul 28 13:30:47.216237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.216127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" event={"ID":"8eca1682-71c7-4d99-a0ff-1be336d28b88","Type":"ContainerDied","Data":"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"}
Jul 28 13:30:47.216237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.216162    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2" event={"ID":"8eca1682-71c7-4d99-a0ff-1be336d28b88","Type":"ContainerDied","Data":"27ad23966989e70a6e089a134646a48525e2d597d74387fbce930bc7b2d04c17"}
Jul 28 13:30:47.216237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.216181    2562 scope.go:117] "RemoveContainer" containerID="91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"
Jul 28 13:30:47.225312 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.225295    2562 scope.go:117] "RemoveContainer" containerID="91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"
Jul 28 13:30:47.225570 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:30:47.225553    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d\": container with ID starting with 91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d not found: ID does not exist" containerID="91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"
Jul 28 13:30:47.225617 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.225577    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d"} err="failed to get container status \"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d\": rpc error: code = NotFound desc = could not find container \"91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d\": container with ID starting with 91f7b4cfff37a453a5964e7433bfefd6a4d2ec17567d095345c71a0eb4ad1d0d not found: ID does not exist"
Jul 28 13:30:47.236709 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.236683    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:30:47.240044 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.240020    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d82b76/test-model-d82b76-kserve-7c55d8f867-d4rs2"]
Jul 28 13:30:47.252291 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.252266    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" path="/var/lib/kubelet/pods/6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9/volumes"
Jul 28 13:30:47.252653 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:47.252640    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eca1682-71c7-4d99-a0ff-1be336d28b88" path="/var/lib/kubelet/pods/8eca1682-71c7-4d99-a0ff-1be336d28b88/volumes"
Jul 28 13:30:49.856343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856309    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"]
Jul 28 13:30:49.856729 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856712    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eca1682-71c7-4d99-a0ff-1be336d28b88" containerName="main"
Jul 28 13:30:49.856729 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856723    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eca1682-71c7-4d99-a0ff-1be336d28b88" containerName="main"
Jul 28 13:30:49.856822 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856766    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" containerName="main"
Jul 28 13:30:49.856822 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856776    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" containerName="main"
Jul 28 13:30:49.856883 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856854    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e84a31b-c4c4-4ab8-97c1-61fb7ceb30a9" containerName="main"
Jul 28 13:30:49.856883 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.856867    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eca1682-71c7-4d99-a0ff-1be336d28b88" containerName="main"
Jul 28 13:30:49.861201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.861184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:49.863962 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.863935    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:30:49.864134 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.863936    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:30:49.867113 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.867087    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"]
Jul 28 13:30:49.916899 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:49.916856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk\" (UID: \"8bf02e6d-dec7-41e4-9716-3a30d828dc05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:50.018425 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:50.018393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk\" (UID: \"8bf02e6d-dec7-41e4-9716-3a30d828dc05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:50.019129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:50.019106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk\" (UID: \"8bf02e6d-dec7-41e4-9716-3a30d828dc05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:50.173568 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:50.173464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:50.312806 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:50.312772    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"]
Jul 28 13:30:50.315907 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:30:50.315880    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf02e6d_dec7_41e4_9716_3a30d828dc05.slice/crio-1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3 WatchSource:0}: Error finding container 1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3: Status 404 returned error can't find the container with id 1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3
Jul 28 13:30:51.239339 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:51.239302    2562 generic.go:356] "Generic (PLEG): container finished" podID="8bf02e6d-dec7-41e4-9716-3a30d828dc05" containerID="7280f91d0fc9b796a5ff82b1ed1e2b6888148d8f3fc62c631eaac59ed18ba1d5" exitCode=0
Jul 28 13:30:51.239703 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:51.239384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk" event={"ID":"8bf02e6d-dec7-41e4-9716-3a30d828dc05","Type":"ContainerDied","Data":"7280f91d0fc9b796a5ff82b1ed1e2b6888148d8f3fc62c631eaac59ed18ba1d5"}
Jul 28 13:30:51.239703 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:51.239439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk" event={"ID":"8bf02e6d-dec7-41e4-9716-3a30d828dc05","Type":"ContainerStarted","Data":"1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3"}
Jul 28 13:30:52.087124 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:52.087092    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:30:52.372588 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:52.372564    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:52.439787 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:52.439753    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca\") pod \"8bf02e6d-dec7-41e4-9716-3a30d828dc05\" (UID: \"8bf02e6d-dec7-41e4-9716-3a30d828dc05\") "
Jul 28 13:30:52.440144 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:52.440119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8bf02e6d-dec7-41e4-9716-3a30d828dc05" (UID: "8bf02e6d-dec7-41e4-9716-3a30d828dc05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:30:52.540681 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:52.540645    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf02e6d-dec7-41e4-9716-3a30d828dc05-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:30:53.250210 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:53.250184    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"
Jul 28 13:30:53.252284 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:53.252257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk" event={"ID":"8bf02e6d-dec7-41e4-9716-3a30d828dc05","Type":"ContainerDied","Data":"1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3"}
Jul 28 13:30:53.252421 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:30:53.252298    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1048d23efc5b0b06eccb74dbe4cd8cd6d3526491624fd2dbf0ea6c42322f87f3"
Jul 28 13:31:00.658457 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:00.658390    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"]
Jul 28 13:31:00.661078 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:00.661050    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d82b76-rv6tk"]
Jul 28 13:31:01.251899 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:01.251866    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf02e6d-dec7-41e4-9716-3a30d828dc05" path="/var/lib/kubelet/pods/8bf02e6d-dec7-41e4-9716-3a30d828dc05/volumes"
Jul 28 13:31:03.588626 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:03.588587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:31:07.566865 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:07.566824    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:31:07.567355 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:07.567091    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy" containerID="cri-o://8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2" gracePeriod=30
Jul 28 13:31:12.824243 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.824213    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:31:12.930657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930563    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.930657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930613    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.930657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.930657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930653    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpzl4\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930902    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930957    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token\") pod \"7dc20be5-f178-46ee-b372-a566d6028dbc\" (UID: \"7dc20be5-f178-46ee-b372-a566d6028dbc\") "
Jul 28 13:31:12.931015 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.930971    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931316    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931334    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc20be5-f178-46ee-b372-a566d6028dbc-istiod-ca-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931346    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-workload-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:12.931363 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931355    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-credential-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:12.931833 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.931688    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:31:12.933422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.933388    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:31:12.933552 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.933444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:31:12.933788 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.933764    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:31:12.933788 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:12.933772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4" (OuterVolumeSpecName: "kube-api-access-dpzl4") pod "7dc20be5-f178-46ee-b372-a566d6028dbc" (UID: "7dc20be5-f178-46ee-b372-a566d6028dbc"). InnerVolumeSpecName "kube-api-access-dpzl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:31:13.032293 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.032255    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpzl4\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-kube-api-access-dpzl4\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:13.032293 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.032288    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-podinfo\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:13.032293 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.032300    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-envoy\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:13.032540 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.032313    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-token\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:13.032540 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.032324    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc20be5-f178-46ee-b372-a566d6028dbc-istio-data\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:31:13.342707 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.342665    2562 generic.go:356] "Generic (PLEG): container finished" podID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerID="8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2" exitCode=0
Jul 28 13:31:13.342884 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.342730    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"
Jul 28 13:31:13.342884 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.342758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerDied","Data":"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"}
Jul 28 13:31:13.342884 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.342817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x" event={"ID":"7dc20be5-f178-46ee-b372-a566d6028dbc","Type":"ContainerDied","Data":"4385bd2192dd4d5567a4f4be414ba763a8a8d0658239cc2e9847e8710daa835d"}
Jul 28 13:31:13.342884 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.342841    2562 scope.go:117] "RemoveContainer" containerID="8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"
Jul 28 13:31:13.352064 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.352032    2562 scope.go:117] "RemoveContainer" containerID="c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"
Jul 28 13:31:13.359265 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.359240    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:31:13.361339 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.361301    2562 scope.go:117] "RemoveContainer" containerID="8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"
Jul 28 13:31:13.361945 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:31:13.361727    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2\": container with ID starting with 8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2 not found: ID does not exist" containerID="8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"
Jul 28 13:31:13.361945 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.361790    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2"} err="failed to get container status \"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2\": rpc error: code = NotFound desc = could not find container \"8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2\": container with ID starting with 8d7c190d3dd9ea45b616db7d581911d0aa0ddf243e32033688d935ac0f2117e2 not found: ID does not exist"
Jul 28 13:31:13.361945 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.361817    2562 scope.go:117] "RemoveContainer" containerID="c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"
Jul 28 13:31:13.362204 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:31:13.362176    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02\": container with ID starting with c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02 not found: ID does not exist" containerID="c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"
Jul 28 13:31:13.362265 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.362214    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02"} err="failed to get container status \"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02\": rpc error: code = NotFound desc = could not find container \"c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02\": container with ID starting with c4a68de12beaf63bcfe766205fc7ac3ffec6433043194702cd5fb520bb3adc02 not found: ID does not exist"
Jul 28 13:31:13.364289 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:13.364271    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d82b76-openshift-default-59d869f4d6-76h8x"]
Jul 28 13:31:15.251653 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:15.251613    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" path="/var/lib/kubelet/pods/7dc20be5-f178-46ee-b372-a566d6028dbc/volumes"
Jul 28 13:31:17.990117 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:17.990074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:31:37.615198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615157    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615528    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf02e6d-dec7-41e4-9716-3a30d828dc05" containerName="revoke-keys"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615542    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf02e6d-dec7-41e4-9716-3a30d828dc05" containerName="revoke-keys"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615563    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615587    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615618    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615628    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615701    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615709    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dc20be5-f178-46ee-b372-a566d6028dbc" containerName="istio-proxy"
Jul 28 13:31:37.615814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.615718    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf02e6d-dec7-41e4-9716-3a30d828dc05" containerName="revoke-keys"
Jul 28 13:31:37.618700 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.618678    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.621161 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.621138    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 13:31:37.621281 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.621231    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-44e001-openshift-default-dockercfg-l8wfj\""
Jul 28 13:31:37.632230 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.632206    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:31:37.654285 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654285 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6sq4\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654722 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.654722 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.654582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755448 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6sq4\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.755656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756098 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.755699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756098 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.756009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.756144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.756155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756342 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.756317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.756529 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.756511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.758147 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.758126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.758378 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.758358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.769270 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.769241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.769517 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.769496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6sq4\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4\") pod \"e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") " pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:37.929606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:37.929503    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:38.069817 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.069791    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:31:38.072145 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:31:38.072120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ede725_9a33_46e6_be5e_13ce490d4e5b.slice/crio-26eb82bfd20edfacd15d86703c69b695dae56fae788529a212d0a6cf867c4969 WatchSource:0}: Error finding container 26eb82bfd20edfacd15d86703c69b695dae56fae788529a212d0a6cf867c4969: Status 404 returned error can't find the container with id 26eb82bfd20edfacd15d86703c69b695dae56fae788529a212d0a6cf867c4969
Jul 28 13:31:38.074284 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.074250    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:38.074368 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.074315    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:38.074368 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.074344    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:38.464082 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.464038    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerStarted","Data":"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"}
Jul 28 13:31:38.464258 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.464100    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerStarted","Data":"26eb82bfd20edfacd15d86703c69b695dae56fae788529a212d0a6cf867c4969"}
Jul 28 13:31:38.484996 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.484935    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podStartSLOduration=1.484916004 podStartE2EDuration="1.484916004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:31:38.482062816 +0000 UTC m=+2411.799081805" watchObservedRunningTime="2026-07-28 13:31:38.484916004 +0000 UTC m=+2411.801934994"
Jul 28 13:31:38.930108 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:38.930047    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:39.935030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:39.934996    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:40.473223 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:40.473185    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:40.474224 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:40.474202    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:49.885386 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:49.885301    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:31:56.514940 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:56.514890    2562 patch_prober.go:28] interesting pod/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 13:31:56.515347 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:56.514985    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 13:31:58.565253 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.565218    2562 generic.go:356] "Generic (PLEG): container finished" podID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerID="e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81" exitCode=0
Jul 28 13:31:58.565791 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.565272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerDied","Data":"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"}
Jul 28 13:31:58.565868 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.565807    2562 scope.go:117] "RemoveContainer" containerID="e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"
Jul 28 13:31:58.566911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.566870    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:58.566988 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.566951    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:58.567032 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:58.566989    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:31:59.571448 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:59.571408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerStarted","Data":"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"}
Jul 28 13:31:59.930354 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:59.930261    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:31:59.930583 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:59.930562    2562 patch_prober.go:28] interesting pod/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 28 13:31:59.930627 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:31:59.930611    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 28 13:32:00.930878 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:00.930837    2562 patch_prober.go:28] interesting pod/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 28 13:32:00.931295 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:00.930896    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 28 13:32:01.930229 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:01.930190    2562 patch_prober.go:28] interesting pod/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 28 13:32:01.930400 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:01.930260    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 28 13:32:02.930755 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:02.930710    2562 patch_prober.go:28] interesting pod/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 28 13:32:02.931247 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:02.930809    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 28 13:32:03.934082 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:03.934044    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:32:03.934516 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:03.934315    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:32:03.935074 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:03.935058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:32:08.729433 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.729401    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"]
Jul 28 13:32:08.732821 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.732801    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:08.739769 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.736120    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:32:08.739769 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.736155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:32:08.743027 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.742987    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"]
Jul 28 13:32:08.768079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.768044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-955728-kkjbz\" (UID: \"fac4b836-ed67-4a99-be78-bc6c1e8dadce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:08.869059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.869007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-955728-kkjbz\" (UID: \"fac4b836-ed67-4a99-be78-bc6c1e8dadce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:08.869655 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:08.869635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-955728-kkjbz\" (UID: \"fac4b836-ed67-4a99-be78-bc6c1e8dadce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:09.049636 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:09.049591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:09.184549 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:09.184517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"]
Jul 28 13:32:09.186814 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:32:09.186779    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac4b836_ed67_4a99_be78_bc6c1e8dadce.slice/crio-3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb WatchSource:0}: Error finding container 3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb: Status 404 returned error can't find the container with id 3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb
Jul 28 13:32:09.619761 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:09.619703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz" event={"ID":"fac4b836-ed67-4a99-be78-bc6c1e8dadce","Type":"ContainerStarted","Data":"e76cf8690edca87c91cd6c8ab2b6a78f09505a5d0149a51b00dd3582253a5964"}
Jul 28 13:32:09.619941 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:09.619782    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz" event={"ID":"fac4b836-ed67-4a99-be78-bc6c1e8dadce","Type":"ContainerStarted","Data":"3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb"}
Jul 28 13:32:09.636701 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:09.636647    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz" podStartSLOduration=1.6366315949999999 podStartE2EDuration="1.636631595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:32:09.634341457 +0000 UTC m=+2442.951360448" watchObservedRunningTime="2026-07-28 13:32:09.636631595 +0000 UTC m=+2442.953650585"
Jul 28 13:32:10.626622 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:10.626584    2562 generic.go:356] "Generic (PLEG): container finished" podID="fac4b836-ed67-4a99-be78-bc6c1e8dadce" containerID="e76cf8690edca87c91cd6c8ab2b6a78f09505a5d0149a51b00dd3582253a5964" exitCode=0
Jul 28 13:32:10.627041 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:10.626664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz" event={"ID":"fac4b836-ed67-4a99-be78-bc6c1e8dadce","Type":"ContainerDied","Data":"e76cf8690edca87c91cd6c8ab2b6a78f09505a5d0149a51b00dd3582253a5964"}
Jul 28 13:32:11.768408 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:11.768381    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:11.794636 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:11.794606    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca\") pod \"fac4b836-ed67-4a99-be78-bc6c1e8dadce\" (UID: \"fac4b836-ed67-4a99-be78-bc6c1e8dadce\") "
Jul 28 13:32:11.794965 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:11.794940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fac4b836-ed67-4a99-be78-bc6c1e8dadce" (UID: "fac4b836-ed67-4a99-be78-bc6c1e8dadce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:32:11.895402 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:11.895313    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fac4b836-ed67-4a99-be78-bc6c1e8dadce-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:12.639337 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:12.639302    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz" event={"ID":"fac4b836-ed67-4a99-be78-bc6c1e8dadce","Type":"ContainerDied","Data":"3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb"}
Jul 28 13:32:12.639498 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:12.639358    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f4c20676247848dce796a0f4f31c16c3c9788150f9b9c5edd8556f0f68be5cb"
Jul 28 13:32:12.639498 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:12.639313    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"
Jul 28 13:32:24.442285 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:24.442241    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"]
Jul 28 13:32:24.450983 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:24.450951    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-955728-kkjbz"]
Jul 28 13:32:25.251422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:25.251385    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fac4b836-ed67-4a99-be78-bc6c1e8dadce" path="/var/lib/kubelet/pods/fac4b836-ed67-4a99-be78-bc6c1e8dadce/volumes"
Jul 28 13:32:27.382996 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:27.382961    2562 scope.go:117] "RemoveContainer" containerID="1d0d438b869f5b594126212ba3af18a193726450ee37941b4b3a21a3be58f896"
Jul 28 13:32:30.593958 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:30.593922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:32:31.664936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.664901    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"]
Jul 28 13:32:31.665350 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.665276    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac4b836-ed67-4a99-be78-bc6c1e8dadce" containerName="revoke-keys"
Jul 28 13:32:31.665350 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.665288    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac4b836-ed67-4a99-be78-bc6c1e8dadce" containerName="revoke-keys"
Jul 28 13:32:31.665350 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.665342    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fac4b836-ed67-4a99-be78-bc6c1e8dadce" containerName="revoke-keys"
Jul 28 13:32:31.668120 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.668099    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:31.670567 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.670539    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:32:31.670567 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.670552    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:32:31.675286 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.675260    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"]
Jul 28 13:32:31.780576 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.780541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44e001-hzxfg\" (UID: \"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:31.881677 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.881636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44e001-hzxfg\" (UID: \"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:31.882328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.882309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44e001-hzxfg\" (UID: \"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:31.980045 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:31.979959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:32.117245 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:32.117212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"]
Jul 28 13:32:32.120412 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:32:32.120380    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9ee0e5_bc8f_41cd_a1f7_70ea823a1a41.slice/crio-14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86 WatchSource:0}: Error finding container 14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86: Status 404 returned error can't find the container with id 14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86
Jul 28 13:32:32.735062 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:32.734963    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg" event={"ID":"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41","Type":"ContainerStarted","Data":"e683b90c292423cefe94e22d5ead9f5014044b26d60bf444591dce0cf653ce68"}
Jul 28 13:32:32.735062 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:32.735016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg" event={"ID":"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41","Type":"ContainerStarted","Data":"14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86"}
Jul 28 13:32:32.750379 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:32.750326    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg" podStartSLOduration=1.750309298 podStartE2EDuration="1.750309298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:32:32.749993647 +0000 UTC m=+2466.067012651" watchObservedRunningTime="2026-07-28 13:32:32.750309298 +0000 UTC m=+2466.067328287"
Jul 28 13:32:33.740764 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:33.740708    2562 generic.go:356] "Generic (PLEG): container finished" podID="ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" containerID="e683b90c292423cefe94e22d5ead9f5014044b26d60bf444591dce0cf653ce68" exitCode=0
Jul 28 13:32:33.741147 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:33.740799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg" event={"ID":"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41","Type":"ContainerDied","Data":"e683b90c292423cefe94e22d5ead9f5014044b26d60bf444591dce0cf653ce68"}
Jul 28 13:32:34.873806 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:34.873781    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:35.013190 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.013101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca\") pod \"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41\" (UID: \"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41\") "
Jul 28 13:32:35.013442 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.013422    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" (UID: "ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:32:35.114008 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.113972    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:35.751750 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.751719    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"
Jul 28 13:32:35.751952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.751763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg" event={"ID":"ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41","Type":"ContainerDied","Data":"14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86"}
Jul 28 13:32:35.751952 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:35.751811    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14a51c4f164a71107bd2922f7fbb244f5cbbe6dd6a1a8ccaea1a942eee562a86"
Jul 28 13:32:47.348784 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:47.348731    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"]
Jul 28 13:32:47.351190 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:47.351166    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44e001-hzxfg"]
Jul 28 13:32:49.251775 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:49.251725    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" path="/var/lib/kubelet/pods/ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41/volumes"
Jul 28 13:32:52.187561 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:52.187522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:32:54.027498 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:54.027463    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:32:54.027934 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:54.027707    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy" containerID="cri-o://a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668" gracePeriod=30
Jul 28 13:32:59.279170 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.279145    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:32:59.331217 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331130    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6sq4\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331217 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331197    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331223    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331320    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331356    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331392    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331420 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331415    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331731 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331476    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs\") pod \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\" (UID: \"d3ede725-9a33-46e6-be5e-13ce490d4e5b\") "
Jul 28 13:32:59.331731 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331548    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:32:59.331731 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331637    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:32:59.331942 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331795    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:32:59.331942 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331860    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-credential-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.332054 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.331899    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.332054 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.332004    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:32:59.332334 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.332197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:32:59.333661 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.333635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4" (OuterVolumeSpecName: "kube-api-access-b6sq4") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "kube-api-access-b6sq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:32:59.333661 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.333647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:32:59.333832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.333712    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:32:59.334013 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.333997    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3ede725-9a33-46e6-be5e-13ce490d4e5b" (UID: "d3ede725-9a33-46e6-be5e-13ce490d4e5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:32:59.433330 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433290    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-token\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433330 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433321    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-envoy\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433330 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433330    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-podinfo\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433330 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433339    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istiod-ca-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433621 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433348    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-workload-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433621 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433356    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6sq4\" (UniqueName: \"kubernetes.io/projected/d3ede725-9a33-46e6-be5e-13ce490d4e5b-kube-api-access-b6sq4\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.433621 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.433366    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ede725-9a33-46e6-be5e-13ce490d4e5b-istio-data\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:32:59.868538 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.868494    2562 generic.go:356] "Generic (PLEG): container finished" podID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerID="a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668" exitCode=0
Jul 28 13:32:59.868757 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.868561    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"
Jul 28 13:32:59.868757 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.868576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerDied","Data":"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"}
Jul 28 13:32:59.868757 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.868670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk" event={"ID":"d3ede725-9a33-46e6-be5e-13ce490d4e5b","Type":"ContainerDied","Data":"26eb82bfd20edfacd15d86703c69b695dae56fae788529a212d0a6cf867c4969"}
Jul 28 13:32:59.868757 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.868700    2562 scope.go:117] "RemoveContainer" containerID="a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"
Jul 28 13:32:59.878700 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.878683    2562 scope.go:117] "RemoveContainer" containerID="e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"
Jul 28 13:32:59.887926 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.887793    2562 scope.go:117] "RemoveContainer" containerID="a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"
Jul 28 13:32:59.888194 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:32:59.888171    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668\": container with ID starting with a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668 not found: ID does not exist" containerID="a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"
Jul 28 13:32:59.888285 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.888206    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668"} err="failed to get container status \"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668\": rpc error: code = NotFound desc = could not find container \"a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668\": container with ID starting with a63bd080624b2048ac5613227fd3330812c6feaaa6960831f43a302c9b9d5668 not found: ID does not exist"
Jul 28 13:32:59.888285 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.888231    2562 scope.go:117] "RemoveContainer" containerID="e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"
Jul 28 13:32:59.888498 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:32:59.888476    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81\": container with ID starting with e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81 not found: ID does not exist" containerID="e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"
Jul 28 13:32:59.888535 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.888504    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81"} err="failed to get container status \"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81\": rpc error: code = NotFound desc = could not find container \"e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81\": container with ID starting with e606660a712d8de05daaa1a789918d905bf96c25f6372c8a627e5b8a5cb17e81 not found: ID does not exist"
Jul 28 13:32:59.889460 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.889441    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:32:59.895303 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:32:59.895281    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-44e001-openshift-default-54f4dbc4b-69ktk"]
Jul 28 13:33:01.252064 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:01.252031    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" path="/var/lib/kubelet/pods/d3ede725-9a33-46e6-be5e-13ce490d4e5b/volumes"
Jul 28 13:33:04.488079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:04.488041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:33:24.647232 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647141    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:33:24.647722 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647685    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" containerName="revoke-keys"
Jul 28 13:33:24.647722 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647701    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" containerName="revoke-keys"
Jul 28 13:33:24.647869 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647726    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:33:24.647869 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647735    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:33:24.647869 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647860    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:33:24.647999 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647875    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca9ee0e5-bc8f-41cd-a1f7-70ea823a1a41" containerName="revoke-keys"
Jul 28 13:33:24.647999 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.647885    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:33:24.651374 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.651348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.653877 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.653853    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8fd168-openshift-default-dockercfg-bb8c4\""
Jul 28 13:33:24.654029 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.654010    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 13:33:24.660853 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.660825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:33:24.772815 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.772815 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772834    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772854    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbw8l\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.772992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773177 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.773059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773177 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.773088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.773177 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.773107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873699 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873829    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.873914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbw8l\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.873972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874268 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874527 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.874669 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.874650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.876265 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.876241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.876496 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.876475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.881923 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.881902    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.882092 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.882072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbw8l\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l\") pod \"e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") " pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:24.962123 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:24.962032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:25.107381 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:25.107354    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:33:25.109966 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:33:25.109938    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243e9e2d_a329_41e2_9ab9_f64184f2b5aa.slice/crio-9b6a239c6fc1886bda1f31c90253735e99808cb7dd0fe61d89697f9b76713458 WatchSource:0}: Error finding container 9b6a239c6fc1886bda1f31c90253735e99808cb7dd0fe61d89697f9b76713458: Status 404 returned error can't find the container with id 9b6a239c6fc1886bda1f31c90253735e99808cb7dd0fe61d89697f9b76713458
Jul 28 13:33:25.112159 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:25.112115    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:25.112267 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:25.112179    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:25.112267 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:25.112208    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:26.001019 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:26.000984    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerStarted","Data":"36f87724229bf31aee8fc3eeefe859336a7599bcbf79f9184f557bb55832a068"}
Jul 28 13:33:26.001525 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:26.001041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerStarted","Data":"9b6a239c6fc1886bda1f31c90253735e99808cb7dd0fe61d89697f9b76713458"}
Jul 28 13:33:26.026256 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:26.026200    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podStartSLOduration=2.026185297 podStartE2EDuration="2.026185297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:33:26.023671772 +0000 UTC m=+2519.340690763" watchObservedRunningTime="2026-07-28 13:33:26.026185297 +0000 UTC m=+2519.343204287"
Jul 28 13:33:26.962760 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:26.962691    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:26.967832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:26.967807    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:27.006813 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:27.006784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:27.007817 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:27.007796    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:36.492221 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:36.492182    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:33:43.048992 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:43.048942    2562 patch_prober.go:28] interesting pod/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 13:33:43.049425 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:43.049024    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 13:33:44.089436 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.089401    2562 generic.go:356] "Generic (PLEG): container finished" podID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerID="36f87724229bf31aee8fc3eeefe859336a7599bcbf79f9184f557bb55832a068" exitCode=0
Jul 28 13:33:44.089997 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.089481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerDied","Data":"36f87724229bf31aee8fc3eeefe859336a7599bcbf79f9184f557bb55832a068"}
Jul 28 13:33:44.090073 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.090006    2562 scope.go:117] "RemoveContainer" containerID="36f87724229bf31aee8fc3eeefe859336a7599bcbf79f9184f557bb55832a068"
Jul 28 13:33:44.091067 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.091041    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:44.091171 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.091107    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:44.091171 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:44.091138    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:33:45.095530 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:45.095497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerStarted","Data":"d6e5d685f6224ebb335f37fff3160f3f20bb0bc9911f63371e9f07a1593527b0"}
Jul 28 13:33:45.962711 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:45.962677    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:45.963067 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:45.963042    2562 patch_prober.go:28] interesting pod/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 13:33:45.963169 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:45.963101    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 13:33:46.963361 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:46.963316    2562 patch_prober.go:28] interesting pod/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 13:33:46.963868 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:46.963380    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 13:33:47.962600 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:47.962551    2562 patch_prober.go:28] interesting pod/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 13:33:47.962833 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:47.962623    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 13:33:48.965900 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:48.965863    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:48.966331 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:48.966117    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:33:48.966803 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:33:48.966786    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:34:21.602871 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.602826    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:34:21.603756 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.603711    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:34:21.603941 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.603926    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ede725-9a33-46e6-be5e-13ce490d4e5b" containerName="istio-proxy"
Jul 28 13:34:21.608836 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.608804    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.612359 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.612315    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"default-dockercfg-5h6lt\""
Jul 28 13:34:21.613293 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.613267    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"auth-test-model-2c9e23-kserve-self-signed-certs\""
Jul 28 13:34:21.613422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.613292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"openshift-service-ca.crt\""
Jul 28 13:34:21.613422 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.613351    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"kube-root-ca.crt\""
Jul 28 13:34:21.615529 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.615509    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:34:21.718201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.718201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.718432 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.718432 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.718432 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jds8w\" (UniqueName: \"kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.718432 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.718392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819553 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819714 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819714 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jds8w\" (UniqueName: \"kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819714 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819714 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.819714 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.819711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.820044 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.820018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.820098 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.820032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.820140 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.820095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.822069 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.822047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.822377 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.822358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.828081 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.828057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jds8w\" (UniqueName: \"kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w\") pod \"auth-test-model-2c9e23-kserve-589556c979-ldfb2\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") " pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:21.920002 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:21.919905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:22.064005 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:22.063977    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:34:22.066567 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:34:22.066537    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4183f1f_dee4_4e84_9346_a73f9351f68d.slice/crio-f66df787fba05bbb70ae461e6c5e817be3f5830f127e4fc4722ee10853925fb7 WatchSource:0}: Error finding container f66df787fba05bbb70ae461e6c5e817be3f5830f127e4fc4722ee10853925fb7: Status 404 returned error can't find the container with id f66df787fba05bbb70ae461e6c5e817be3f5830f127e4fc4722ee10853925fb7
Jul 28 13:34:22.292315 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:22.292276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" event={"ID":"e4183f1f-dee4-4e84-9346-a73f9351f68d","Type":"ContainerStarted","Data":"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"}
Jul 28 13:34:22.292315 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:22.292317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" event={"ID":"e4183f1f-dee4-4e84-9346-a73f9351f68d","Type":"ContainerStarted","Data":"f66df787fba05bbb70ae461e6c5e817be3f5830f127e4fc4722ee10853925fb7"}
Jul 28 13:34:22.311622 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:22.311566    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" podStartSLOduration=1.31155019 podStartE2EDuration="1.31155019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:34:22.310809368 +0000 UTC m=+2575.627828387" watchObservedRunningTime="2026-07-28 13:34:22.31155019 +0000 UTC m=+2575.628569176"
Jul 28 13:34:24.898523 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:24.898488    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:34:31.920704 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:31.920664    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:31.921264 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:31.920755    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:31.933535 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:31.933497    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:32.351693 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:32.351664    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:34:42.693759 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.693699    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:34:42.697174 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.697154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.699635 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.699600    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"kube-root-ca.crt\""
Jul 28 13:34:42.699793 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.699666    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"openshift-service-ca.crt\""
Jul 28 13:34:42.700695 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.700677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"default-dockercfg-42m8k\""
Jul 28 13:34:42.700820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.700781    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"auth-test-model-8fd168-kserve-self-signed-certs\""
Jul 28 13:34:42.705058 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.704654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:34:42.714115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.714252 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7qvf\" (UniqueName: \"kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.714252 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.714252 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.714356 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.714394 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.714358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815061 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815061 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815326 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815326 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7qvf\" (UniqueName: \"kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815326 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815560 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815560 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815560 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.815679 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.815659    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.817549 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.817530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.817843 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.817826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:42.825618 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:42.825592    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7qvf\" (UniqueName: \"kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf\") pod \"auth-test-model-8fd168-kserve-b4b5585f9-pfnfx\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") " pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:43.008516 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:43.008476    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:43.145774 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:43.145723    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:34:43.148086 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:34:43.148048    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod149281e6_6148_43ce_aa71_e6fc4ca0965e.slice/crio-6d6227e0d4b4ea521fad1570973a598f4bb791011425146b43fa0f9a8ad694dd WatchSource:0}: Error finding container 6d6227e0d4b4ea521fad1570973a598f4bb791011425146b43fa0f9a8ad694dd: Status 404 returned error can't find the container with id 6d6227e0d4b4ea521fad1570973a598f4bb791011425146b43fa0f9a8ad694dd
Jul 28 13:34:43.396169 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:43.396076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" event={"ID":"149281e6-6148-43ce-aa71-e6fc4ca0965e","Type":"ContainerStarted","Data":"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"}
Jul 28 13:34:43.396169 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:43.396117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" event={"ID":"149281e6-6148-43ce-aa71-e6fc4ca0965e","Type":"ContainerStarted","Data":"6d6227e0d4b4ea521fad1570973a598f4bb791011425146b43fa0f9a8ad694dd"}
Jul 28 13:34:43.413406 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:43.413355    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" podStartSLOduration=1.413340211 podStartE2EDuration="1.413340211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:34:43.412463629 +0000 UTC m=+2596.729482621" watchObservedRunningTime="2026-07-28 13:34:43.413340211 +0000 UTC m=+2596.730359201"
Jul 28 13:34:46.992525 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:46.992479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:34:53.008767 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:53.008716    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:53.008767 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:53.008775    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:53.021768 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:53.021717    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:34:53.456568 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:34:53.456477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:35:37.090515 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:35:37.090481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:36:04.087705 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:36:04.087660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:36:19.994070 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:36:19.993982    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:36:40.682011 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:36:40.681977    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:36:56.492606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:36:56.492557    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:37:17.290568 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:37:17.290530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:37:27.434911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:37:27.434871    2562 scope.go:117] "RemoveContainer" containerID="7280f91d0fc9b796a5ff82b1ed1e2b6888148d8f3fc62c631eaac59ed18ba1d5"
Jul 28 13:37:34.992884 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:37:34.992845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:38:02.597250 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:38:02.597171    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:38:21.596253 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:38:21.596211    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:38:27.455079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:38:27.455046    2562 scope.go:117] "RemoveContainer" containerID="e76cf8690edca87c91cd6c8ab2b6a78f09505a5d0149a51b00dd3582253a5964"
Jul 28 13:38:43.787894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:38:43.787851    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:39:01.218579 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:01.218538    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:39:27.475518 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:27.475436    2562 scope.go:117] "RemoveContainer" containerID="e683b90c292423cefe94e22d5ead9f5014044b26d60bf444591dce0cf653ce68"
Jul 28 13:39:27.893647 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:27.893606    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:39:30.969182 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:30.969146    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:39:31.874557 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:31.874518    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:39:39.030976 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:39.030940    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:39:45.988612 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:45.988561    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" podUID="e4183f1f-dee4-4e84-9346-a73f9351f68d" containerName="main" containerID="cri-o://c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10" gracePeriod=45
Jul 28 13:39:46.243069 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.243041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:39:46.388251 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388467 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388274    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388467 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388309    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388467 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388368    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388467 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388387    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jds8w\" (UniqueName: \"kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388689 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir\") pod \"e4183f1f-dee4-4e84-9346-a73f9351f68d\" (UID: \"e4183f1f-dee4-4e84-9346-a73f9351f68d\") "
Jul 28 13:39:46.388689 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home" (OuterVolumeSpecName: "home") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:46.388689 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388653    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:46.388903 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.388788    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:46.389170 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.389101    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.389170 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.389128    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.389170 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.389143    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.390883 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.390855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm" (OuterVolumeSpecName: "dshm") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:46.390982 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.390893    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:39:46.390982 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.390932    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w" (OuterVolumeSpecName: "kube-api-access-jds8w") pod "e4183f1f-dee4-4e84-9346-a73f9351f68d" (UID: "e4183f1f-dee4-4e84-9346-a73f9351f68d"). InnerVolumeSpecName "kube-api-access-jds8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:39:46.490509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.490472    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jds8w\" (UniqueName: \"kubernetes.io/projected/e4183f1f-dee4-4e84-9346-a73f9351f68d-kube-api-access-jds8w\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.490509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.490500    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4183f1f-dee4-4e84-9346-a73f9351f68d-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.490509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.490510    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4183f1f-dee4-4e84-9346-a73f9351f68d-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:46.887413 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.887371    2562 generic.go:356] "Generic (PLEG): container finished" podID="e4183f1f-dee4-4e84-9346-a73f9351f68d" containerID="c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10" exitCode=0
Jul 28 13:39:46.887628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.887420    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"
Jul 28 13:39:46.887628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.887445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" event={"ID":"e4183f1f-dee4-4e84-9346-a73f9351f68d","Type":"ContainerDied","Data":"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"}
Jul 28 13:39:46.887628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.887485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2" event={"ID":"e4183f1f-dee4-4e84-9346-a73f9351f68d","Type":"ContainerDied","Data":"f66df787fba05bbb70ae461e6c5e817be3f5830f127e4fc4722ee10853925fb7"}
Jul 28 13:39:46.887628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.887506    2562 scope.go:117] "RemoveContainer" containerID="c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"
Jul 28 13:39:46.894177 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.894141    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" podUID="149281e6-6148-43ce-aa71-e6fc4ca0965e" containerName="main" containerID="cri-o://bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596" gracePeriod=45
Jul 28 13:39:46.897533 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.897512    2562 scope.go:117] "RemoveContainer" containerID="c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"
Jul 28 13:39:46.897803 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:39:46.897782    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10\": container with ID starting with c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10 not found: ID does not exist" containerID="c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"
Jul 28 13:39:46.897860 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.897812    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10"} err="failed to get container status \"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10\": rpc error: code = NotFound desc = could not find container \"c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10\": container with ID starting with c993ea6122105ef576c721c3f932f22d742e91409daa7807a28c07667c1fae10 not found: ID does not exist"
Jul 28 13:39:46.916590 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.916567    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:39:46.925041 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:46.925015    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/auth-test-model-2c9e23-kserve-589556c979-ldfb2"]
Jul 28 13:39:47.151109 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.151034    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:39:47.252106 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.252073    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4183f1f-dee4-4e84-9346-a73f9351f68d" path="/var/lib/kubelet/pods/e4183f1f-dee4-4e84-9346-a73f9351f68d/volumes"
Jul 28 13:39:47.297145 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297366 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297164    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297366 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297205    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297366 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297242    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297366 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297574 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297388    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7qvf\" (UniqueName: \"kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf\") pod \"149281e6-6148-43ce-aa71-e6fc4ca0965e\" (UID: \"149281e6-6148-43ce-aa71-e6fc4ca0965e\") "
Jul 28 13:39:47.297574 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:47.297574 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297542    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache" (OuterVolumeSpecName: "model-cache") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:47.297763 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297626    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home" (OuterVolumeSpecName: "home") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:47.297832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297762    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.297832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297782    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.297832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.297790    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.299858 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.299829    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf" (OuterVolumeSpecName: "kube-api-access-r7qvf") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "kube-api-access-r7qvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:39:47.299970 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.299877    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm" (OuterVolumeSpecName: "dshm") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:39:47.299970 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.299885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "149281e6-6148-43ce-aa71-e6fc4ca0965e" (UID: "149281e6-6148-43ce-aa71-e6fc4ca0965e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:39:47.398579 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.398541    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/149281e6-6148-43ce-aa71-e6fc4ca0965e-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.398794 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.398651    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/149281e6-6148-43ce-aa71-e6fc4ca0965e-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.398794 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.398672    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7qvf\" (UniqueName: \"kubernetes.io/projected/149281e6-6148-43ce-aa71-e6fc4ca0965e-kube-api-access-r7qvf\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:39:47.894688 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.894647    2562 generic.go:356] "Generic (PLEG): container finished" podID="149281e6-6148-43ce-aa71-e6fc4ca0965e" containerID="bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596" exitCode=0
Jul 28 13:39:47.894688 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.894692    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"
Jul 28 13:39:47.894991 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.894770    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" event={"ID":"149281e6-6148-43ce-aa71-e6fc4ca0965e","Type":"ContainerDied","Data":"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"}
Jul 28 13:39:47.894991 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.894817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx" event={"ID":"149281e6-6148-43ce-aa71-e6fc4ca0965e","Type":"ContainerDied","Data":"6d6227e0d4b4ea521fad1570973a598f4bb791011425146b43fa0f9a8ad694dd"}
Jul 28 13:39:47.894991 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.894840    2562 scope.go:117] "RemoveContainer" containerID="bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"
Jul 28 13:39:47.904570 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.904552    2562 scope.go:117] "RemoveContainer" containerID="bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"
Jul 28 13:39:47.904914 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:39:47.904894    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596\": container with ID starting with bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596 not found: ID does not exist" containerID="bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"
Jul 28 13:39:47.904995 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.904924    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596"} err="failed to get container status \"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596\": rpc error: code = NotFound desc = could not find container \"bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596\": container with ID starting with bbcc4fc59397ce4ff8da51313e49aa35c23d9f8b2168b49d5e4975c13d799596 not found: ID does not exist"
Jul 28 13:39:47.918620 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.918596    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:39:47.925803 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:47.925777    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/auth-test-model-8fd168-kserve-b4b5585f9-pfnfx"]
Jul 28 13:39:49.251564 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:49.251529    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149281e6-6148-43ce-aa71-e6fc4ca0965e" path="/var/lib/kubelet/pods/149281e6-6148-43ce-aa71-e6fc4ca0965e/volumes"
Jul 28 13:39:49.425557 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:39:49.425522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:40:01.538197 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538157    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:40:01.538694 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538589    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149281e6-6148-43ce-aa71-e6fc4ca0965e" containerName="main"
Jul 28 13:40:01.538694 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538601    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="149281e6-6148-43ce-aa71-e6fc4ca0965e" containerName="main"
Jul 28 13:40:01.538694 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538611    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4183f1f-dee4-4e84-9346-a73f9351f68d" containerName="main"
Jul 28 13:40:01.538694 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538616    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4183f1f-dee4-4e84-9346-a73f9351f68d" containerName="main"
Jul 28 13:40:01.538913 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538699    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="149281e6-6148-43ce-aa71-e6fc4ca0965e" containerName="main"
Jul 28 13:40:01.538913 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.538715    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4183f1f-dee4-4e84-9346-a73f9351f68d" containerName="main"
Jul 28 13:40:01.543352 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.543327    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.548496 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.548468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"sub-test-model-8fd168-kserve-self-signed-certs\""
Jul 28 13:40:01.549198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.549175    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"openshift-service-ca.crt\""
Jul 28 13:40:01.549343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.549174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"default-dockercfg-42m8k\""
Jul 28 13:40:01.549343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.549212    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"kube-root-ca.crt\""
Jul 28 13:40:01.561913 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.561884    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:40:01.639645 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.639645 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.639911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639689    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.639911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.639911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbdrd\" (UniqueName: \"kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.639911 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.639848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.740984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.740947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.740991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbdrd\" (UniqueName: \"kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741303 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741424 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.741564 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.741545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.743623 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.743605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.743895 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.743877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.754148 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.754120    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbdrd\" (UniqueName: \"kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd\") pod \"sub-test-model-8fd168-kserve-94bfd7c8f-lvd49\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") " pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:01.852354 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:01.852250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:02.010057 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:02.010017    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:40:02.013136 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:40:02.013107    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60d7dbef_a42a_4cfe_b472_2dae4646cbab.slice/crio-86d518baada3f2e6d3a58711d26e7bb5b0e76e977ca23ee1371321df2b571945 WatchSource:0}: Error finding container 86d518baada3f2e6d3a58711d26e7bb5b0e76e977ca23ee1371321df2b571945: Status 404 returned error can't find the container with id 86d518baada3f2e6d3a58711d26e7bb5b0e76e977ca23ee1371321df2b571945
Jul 28 13:40:02.970115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:02.970077    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" event={"ID":"60d7dbef-a42a-4cfe-b472-2dae4646cbab","Type":"ContainerStarted","Data":"bc02d63874ab933e070ca999c346740eedfce7bb9611e7710c2a08861206b30d"}
Jul 28 13:40:02.970501 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:02.970123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" event={"ID":"60d7dbef-a42a-4cfe-b472-2dae4646cbab","Type":"ContainerStarted","Data":"86d518baada3f2e6d3a58711d26e7bb5b0e76e977ca23ee1371321df2b571945"}
Jul 28 13:40:03.003237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:03.003165    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" podStartSLOduration=2.003144665 podStartE2EDuration="2.003144665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:40:03.002993892 +0000 UTC m=+2916.320012899" watchObservedRunningTime="2026-07-28 13:40:03.003144665 +0000 UTC m=+2916.320163657"
Jul 28 13:40:05.606587 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:05.606544    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:40:11.852692 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:11.852639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:11.853332 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:11.852712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:11.865311 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:11.865279    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:12.024528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:12.024486    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:40:51.146335 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:40:51.146258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:41:10.726502 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:10.726464    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:41:18.137088 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:18.137053    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:41:24.032877 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.032847    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:41:24.036306 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.036287    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.039067 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.039042    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"kube-root-ca.crt\""
Jul 28 13:41:24.039244 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.039127    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"default-dockercfg-5h6lt\""
Jul 28 13:41:24.040132 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.040109    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"rate-test-model-2c9e23-kserve-self-signed-certs\""
Jul 28 13:41:24.043972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.043951    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2c9e23\"/\"openshift-service-ca.crt\""
Jul 28 13:41:24.053360 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.053339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:41:24.188571 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlx9g\" (UniqueName: \"kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.188571 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.188867 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.188867 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.188867 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.188867 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.188845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290380 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290380 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290380 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlx9g\" (UniqueName: \"kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290907 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.290907 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.291101 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.290964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.292854 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.292835    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.293048 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.293033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.300617 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.300577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlx9g\" (UniqueName: \"kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g\") pod \"rate-test-model-2c9e23-kserve-c75974cbc-ftzrc\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") " pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.346580 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.346544    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:24.484693 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:24.484664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:41:24.486868 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:41:24.486838    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod003cb8de_0fe7_4812_8e58_da4a8a4a434c.slice/crio-2a7ca3503ce7af2e802f2d13733628f30762def5f449e650da8953272c603e12 WatchSource:0}: Error finding container 2a7ca3503ce7af2e802f2d13733628f30762def5f449e650da8953272c603e12: Status 404 returned error can't find the container with id 2a7ca3503ce7af2e802f2d13733628f30762def5f449e650da8953272c603e12
Jul 28 13:41:25.355638 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:25.355594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" event={"ID":"003cb8de-0fe7-4812-8e58-da4a8a4a434c","Type":"ContainerStarted","Data":"d703895829035b44d8a02d94d19ba941c1b36599a741436185aa39248133341a"}
Jul 28 13:41:25.356061 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:25.355649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" event={"ID":"003cb8de-0fe7-4812-8e58-da4a8a4a434c","Type":"ContainerStarted","Data":"2a7ca3503ce7af2e802f2d13733628f30762def5f449e650da8953272c603e12"}
Jul 28 13:41:25.377062 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:25.377008    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" podStartSLOduration=1.376990786 podStartE2EDuration="1.376990786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:41:25.375284758 +0000 UTC m=+2998.692303749" watchObservedRunningTime="2026-07-28 13:41:25.376990786 +0000 UTC m=+2998.694009808"
Jul 28 13:41:29.002690 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:29.002653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:41:33.156396 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.156327    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" podUID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" containerName="main" containerID="cri-o://bc02d63874ab933e070ca999c346740eedfce7bb9611e7710c2a08861206b30d" gracePeriod=45
Jul 28 13:41:33.392148 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.392118    2562 generic.go:356] "Generic (PLEG): container finished" podID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" containerID="bc02d63874ab933e070ca999c346740eedfce7bb9611e7710c2a08861206b30d" exitCode=0
Jul 28 13:41:33.392288 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.392190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" event={"ID":"60d7dbef-a42a-4cfe-b472-2dae4646cbab","Type":"ContainerDied","Data":"bc02d63874ab933e070ca999c346740eedfce7bb9611e7710c2a08861206b30d"}
Jul 28 13:41:33.408846 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.408785    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:41:33.479928 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.479893    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480104 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.479961    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbdrd\" (UniqueName: \"kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480104 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480104 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480104 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480299 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480104    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs\") pod \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\" (UID: \"60d7dbef-a42a-4cfe-b472-2dae4646cbab\") "
Jul 28 13:41:33.480299 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:41:33.480484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home" (OuterVolumeSpecName: "home") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:41:33.480484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480338    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache" (OuterVolumeSpecName: "model-cache") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:41:33.480484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480444    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:33.480484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480461    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:33.480681 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.480488    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:33.482343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.482305    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm" (OuterVolumeSpecName: "dshm") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:41:33.482453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.482351    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:41:33.482453 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.482364    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd" (OuterVolumeSpecName: "kube-api-access-kbdrd") pod "60d7dbef-a42a-4cfe-b472-2dae4646cbab" (UID: "60d7dbef-a42a-4cfe-b472-2dae4646cbab"). InnerVolumeSpecName "kube-api-access-kbdrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:41:33.580972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.580941    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbdrd\" (UniqueName: \"kubernetes.io/projected/60d7dbef-a42a-4cfe-b472-2dae4646cbab-kube-api-access-kbdrd\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:33.580972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.580969    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d7dbef-a42a-4cfe-b472-2dae4646cbab-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:33.580972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:33.580978    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d7dbef-a42a-4cfe-b472-2dae4646cbab-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:41:34.347003 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.346962    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:34.347466 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.347026    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:34.359801 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.359775    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:34.398162 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.398131    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"
Jul 28 13:41:34.398349 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.398171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49" event={"ID":"60d7dbef-a42a-4cfe-b472-2dae4646cbab","Type":"ContainerDied","Data":"86d518baada3f2e6d3a58711d26e7bb5b0e76e977ca23ee1371321df2b571945"}
Jul 28 13:41:34.398349 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.398225    2562 scope.go:117] "RemoveContainer" containerID="bc02d63874ab933e070ca999c346740eedfce7bb9611e7710c2a08861206b30d"
Jul 28 13:41:34.411113 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.411090    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:41:34.421632 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.421604    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:41:34.429511 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:34.429470    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/sub-test-model-8fd168-kserve-94bfd7c8f-lvd49"]
Jul 28 13:41:35.251393 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:35.251360    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" path="/var/lib/kubelet/pods/60d7dbef-a42a-4cfe-b472-2dae4646cbab/volumes"
Jul 28 13:41:48.799081 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.799042    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:41:48.799771 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.799723    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" containerName="main"
Jul 28 13:41:48.799771 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.799767    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" containerName="main"
Jul 28 13:41:48.799943 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.799854    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d7dbef-a42a-4cfe-b472-2dae4646cbab" containerName="main"
Jul 28 13:41:48.802924 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.802907    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.806873 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.806844    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"openshift-service-ca.crt\""
Jul 28 13:41:48.806982 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.806890    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"rate-test-model-8fd168-kserve-self-signed-certs\""
Jul 28 13:41:48.806982 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.806930    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"default-dockercfg-42m8k\""
Jul 28 13:41:48.806982 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.806941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8fd168\"/\"kube-root-ca.crt\""
Jul 28 13:41:48.813313 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.813285    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:41:48.920229 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr6g8\" (UniqueName: \"kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.920411 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.920411 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.920411 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.920411 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:48.920411 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:48.920394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021656 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr6g8\" (UniqueName: \"kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.021894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.021840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.022191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.022170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.022251 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.022210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.022251 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.022237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.024174 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.024154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.024345 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.024328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.032405 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.032378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr6g8\" (UniqueName: \"kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8\") pod \"rate-test-model-8fd168-kserve-79cb78df95-pb4vk\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") " pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.113295 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.113198    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:49.253543 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.253511    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:41:49.256012 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:41:49.255980    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod638922aa_f562_4d15_85cf_1ee069ed2043.slice/crio-a1361a392c012c0c71f359b006fb091fa65e35531b9b1615d2d152232ef7da27 WatchSource:0}: Error finding container a1361a392c012c0c71f359b006fb091fa65e35531b9b1615d2d152232ef7da27: Status 404 returned error can't find the container with id a1361a392c012c0c71f359b006fb091fa65e35531b9b1615d2d152232ef7da27
Jul 28 13:41:49.472045 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.471952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" event={"ID":"638922aa-f562-4d15-85cf-1ee069ed2043","Type":"ContainerStarted","Data":"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"}
Jul 28 13:41:49.472045 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.471993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" event={"ID":"638922aa-f562-4d15-85cf-1ee069ed2043","Type":"ContainerStarted","Data":"a1361a392c012c0c71f359b006fb091fa65e35531b9b1615d2d152232ef7da27"}
Jul 28 13:41:49.491950 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:49.491896    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" podStartSLOduration=1.491878725 podStartE2EDuration="1.491878725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:41:49.491030968 +0000 UTC m=+3022.808049961" watchObservedRunningTime="2026-07-28 13:41:49.491878725 +0000 UTC m=+3022.808897715"
Jul 28 13:41:52.215233 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:52.215194    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:41:59.114202 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:59.114159    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:59.114721 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:59.114228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:59.126882 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:59.126851    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:41:59.533440 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:41:59.533411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:42:26.698867 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:42:26.698776    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:42:44.700322 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:42:44.700289    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:43:05.303916 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:05.303880    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:43:10.722893 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:10.722854    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:43:11.619139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:11.619106    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:43:12.646800 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.646766    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7002-140696164095792"]
Jul 28 13:43:12.650204 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.650187    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:12.652922 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.652903    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7n25w\""
Jul 28 13:43:12.657276 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.657252    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164095792"]
Jul 28 13:43:12.708347 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.708309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg8cj\" (UniqueName: \"kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj\") pod \"test-curl-7002-140696164095792\" (UID: \"147f01b5-40a8-434e-9bd5-4f7c04dc655a\") " pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:12.809121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.809078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg8cj\" (UniqueName: \"kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj\") pod \"test-curl-7002-140696164095792\" (UID: \"147f01b5-40a8-434e-9bd5-4f7c04dc655a\") " pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:12.817878 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.817850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg8cj\" (UniqueName: \"kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj\") pod \"test-curl-7002-140696164095792\" (UID: \"147f01b5-40a8-434e-9bd5-4f7c04dc655a\") " pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:12.960791 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:12.960680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:13.099398 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:13.099366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164095792"]
Jul 28 13:43:13.102847 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:13.102813    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod147f01b5_40a8_434e_9bd5_4f7c04dc655a.slice/crio-dd0883d4f20cbddf3ab0bf6d286d50f64cb09dbf69d067dfa604cf05e6040cfd WatchSource:0}: Error finding container dd0883d4f20cbddf3ab0bf6d286d50f64cb09dbf69d067dfa604cf05e6040cfd: Status 404 returned error can't find the container with id dd0883d4f20cbddf3ab0bf6d286d50f64cb09dbf69d067dfa604cf05e6040cfd
Jul 28 13:43:13.104161 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:13.104144    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:43:13.870855 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:13.870824    2562 generic.go:356] "Generic (PLEG): container finished" podID="147f01b5-40a8-434e-9bd5-4f7c04dc655a" containerID="21fe86fdd92bc719dc346c4d3bd0886bd6952b0b8e54b27de7a6f842bce24e79" exitCode=0
Jul 28 13:43:13.871166 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:13.870905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164095792" event={"ID":"147f01b5-40a8-434e-9bd5-4f7c04dc655a","Type":"ContainerDied","Data":"21fe86fdd92bc719dc346c4d3bd0886bd6952b0b8e54b27de7a6f842bce24e79"}
Jul 28 13:43:13.871166 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:13.870946    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164095792" event={"ID":"147f01b5-40a8-434e-9bd5-4f7c04dc655a","Type":"ContainerStarted","Data":"dd0883d4f20cbddf3ab0bf6d286d50f64cb09dbf69d067dfa604cf05e6040cfd"}
Jul 28 13:43:15.005657 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.005631    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:15.046806 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.046774    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7002-140696164095792_147f01b5-40a8-434e-9bd5-4f7c04dc655a/test-curl-7002-140696164095792/0.log"
Jul 28 13:43:15.071425 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.071389    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7002-140696164095792"]
Jul 28 13:43:15.076122 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.076098    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7002-140696164095792"]
Jul 28 13:43:15.132769 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.132675    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg8cj\" (UniqueName: \"kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj\") pod \"147f01b5-40a8-434e-9bd5-4f7c04dc655a\" (UID: \"147f01b5-40a8-434e-9bd5-4f7c04dc655a\") "
Jul 28 13:43:15.135037 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.135016    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj" (OuterVolumeSpecName: "kube-api-access-tg8cj") pod "147f01b5-40a8-434e-9bd5-4f7c04dc655a" (UID: "147f01b5-40a8-434e-9bd5-4f7c04dc655a"). InnerVolumeSpecName "kube-api-access-tg8cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:15.199107 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.199074    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7002-140696164094640"]
Jul 28 13:43:15.199463 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.199450    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147f01b5-40a8-434e-9bd5-4f7c04dc655a" containerName="test-curl-7002-140696164095792"
Jul 28 13:43:15.199508 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.199465    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="147f01b5-40a8-434e-9bd5-4f7c04dc655a" containerName="test-curl-7002-140696164095792"
Jul 28 13:43:15.199548 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.199538    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="147f01b5-40a8-434e-9bd5-4f7c04dc655a" containerName="test-curl-7002-140696164095792"
Jul 28 13:43:15.202541 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.202524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:15.209765 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.209712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164094640"]
Jul 28 13:43:15.233498 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.233459    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg8cj\" (UniqueName: \"kubernetes.io/projected/147f01b5-40a8-434e-9bd5-4f7c04dc655a-kube-api-access-tg8cj\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:15.252058 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.252030    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="147f01b5-40a8-434e-9bd5-4f7c04dc655a" path="/var/lib/kubelet/pods/147f01b5-40a8-434e-9bd5-4f7c04dc655a/volumes"
Jul 28 13:43:15.334730 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.334692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d8b2\" (UniqueName: \"kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2\") pod \"test-curl-7002-140696164094640\" (UID: \"a3e11e49-3535-4e11-aa30-a0433274f954\") " pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:15.436158 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.436046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d8b2\" (UniqueName: \"kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2\") pod \"test-curl-7002-140696164094640\" (UID: \"a3e11e49-3535-4e11-aa30-a0433274f954\") " pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:15.445199 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.445176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d8b2\" (UniqueName: \"kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2\") pod \"test-curl-7002-140696164094640\" (UID: \"a3e11e49-3535-4e11-aa30-a0433274f954\") " pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:15.513955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.513916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:15.669971 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.669944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164094640"]
Jul 28 13:43:15.672322 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:15.672294    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3e11e49_3535_4e11_aa30_a0433274f954.slice/crio-e32ad0ee8343f03540a3969f175c1026f6f62806fdcb535e32284fc4f476cf3d WatchSource:0}: Error finding container e32ad0ee8343f03540a3969f175c1026f6f62806fdcb535e32284fc4f476cf3d: Status 404 returned error can't find the container with id e32ad0ee8343f03540a3969f175c1026f6f62806fdcb535e32284fc4f476cf3d
Jul 28 13:43:15.882253 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.882215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164094640" event={"ID":"a3e11e49-3535-4e11-aa30-a0433274f954","Type":"ContainerStarted","Data":"e32ad0ee8343f03540a3969f175c1026f6f62806fdcb535e32284fc4f476cf3d"}
Jul 28 13:43:15.883404 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.883384    2562 scope.go:117] "RemoveContainer" containerID="21fe86fdd92bc719dc346c4d3bd0886bd6952b0b8e54b27de7a6f842bce24e79"
Jul 28 13:43:15.883544 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:15.883409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164095792"
Jul 28 13:43:16.890667 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:16.890629    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3e11e49-3535-4e11-aa30-a0433274f954" containerID="879be6f37184fbe8642c41b2b56b635647598d4388a817d5a6a60250d0b4dbb6" exitCode=0
Jul 28 13:43:16.891079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:16.890690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164094640" event={"ID":"a3e11e49-3535-4e11-aa30-a0433274f954","Type":"ContainerDied","Data":"879be6f37184fbe8642c41b2b56b635647598d4388a817d5a6a60250d0b4dbb6"}
Jul 28 13:43:18.032762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.032713    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:18.055428 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.055403    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7002-140696164094640_a3e11e49-3535-4e11-aa30-a0433274f954/test-curl-7002-140696164094640/0.log"
Jul 28 13:43:18.082543 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.082509    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7002-140696164094640"]
Jul 28 13:43:18.087482 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.087451    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7002-140696164094640"]
Jul 28 13:43:18.159328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.159250    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d8b2\" (UniqueName: \"kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2\") pod \"a3e11e49-3535-4e11-aa30-a0433274f954\" (UID: \"a3e11e49-3535-4e11-aa30-a0433274f954\") "
Jul 28 13:43:18.161461 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.161430    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2" (OuterVolumeSpecName: "kube-api-access-9d8b2") pod "a3e11e49-3535-4e11-aa30-a0433274f954" (UID: "a3e11e49-3535-4e11-aa30-a0433274f954"). InnerVolumeSpecName "kube-api-access-9d8b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:18.260452 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.260425    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d8b2\" (UniqueName: \"kubernetes.io/projected/a3e11e49-3535-4e11-aa30-a0433274f954-kube-api-access-9d8b2\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:18.901509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.901474    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:18.901509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:18.901491    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e32ad0ee8343f03540a3969f175c1026f6f62806fdcb535e32284fc4f476cf3d"
Jul 28 13:43:18.911566 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:43:18.911539    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7002-140696164094640\" is forbidden: User \"system:node:ip-10-0-132-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-196.ec2.internal' and this object" podUID="a3e11e49-3535-4e11-aa30-a0433274f954" pod="opendatahub/test-curl-7002-140696164094640"
Jul 28 13:43:19.252538 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:19.252506    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e11e49-3535-4e11-aa30-a0433274f954" path="/var/lib/kubelet/pods/a3e11e49-3535-4e11-aa30-a0433274f954/volumes"
Jul 28 13:43:21.040627 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.040597    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7002-140696164038672"]
Jul 28 13:43:21.041030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.040979    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e11e49-3535-4e11-aa30-a0433274f954" containerName="test-curl-7002-140696164094640"
Jul 28 13:43:21.041030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.040991    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e11e49-3535-4e11-aa30-a0433274f954" containerName="test-curl-7002-140696164094640"
Jul 28 13:43:21.041104 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.041064    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e11e49-3535-4e11-aa30-a0433274f954" containerName="test-curl-7002-140696164094640"
Jul 28 13:43:21.044406 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.044389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:21.047085 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.047067    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7n25w\""
Jul 28 13:43:21.053552 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.053529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164038672"]
Jul 28 13:43:21.089717 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.089687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8j2\" (UniqueName: \"kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2\") pod \"test-curl-7002-140696164038672\" (UID: \"4473b4b1-c7a3-422c-b97f-22218f7d3c75\") " pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:21.190475 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.190434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8j2\" (UniqueName: \"kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2\") pod \"test-curl-7002-140696164038672\" (UID: \"4473b4b1-c7a3-422c-b97f-22218f7d3c75\") " pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:21.206956 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.206919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8j2\" (UniqueName: \"kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2\") pod \"test-curl-7002-140696164038672\" (UID: \"4473b4b1-c7a3-422c-b97f-22218f7d3c75\") " pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:21.354257 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.354175    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:21.490950 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.490910    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164038672"]
Jul 28 13:43:21.492988 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:21.492960    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4473b4b1_c7a3_422c_b97f_22218f7d3c75.slice/crio-fb705a6348a7749175927ff66789dbaa22dc7f745a780ff47e8c6435a6959db9 WatchSource:0}: Error finding container fb705a6348a7749175927ff66789dbaa22dc7f745a780ff47e8c6435a6959db9: Status 404 returned error can't find the container with id fb705a6348a7749175927ff66789dbaa22dc7f745a780ff47e8c6435a6959db9
Jul 28 13:43:21.916794 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.916714    2562 generic.go:356] "Generic (PLEG): container finished" podID="4473b4b1-c7a3-422c-b97f-22218f7d3c75" containerID="db8cd4b47f070e47d5e3923329d030508ef7ecb43cc5c656a9504702298ef2bc" exitCode=0
Jul 28 13:43:21.916919 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.916778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164038672" event={"ID":"4473b4b1-c7a3-422c-b97f-22218f7d3c75","Type":"ContainerDied","Data":"db8cd4b47f070e47d5e3923329d030508ef7ecb43cc5c656a9504702298ef2bc"}
Jul 28 13:43:21.916919 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:21.916859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164038672" event={"ID":"4473b4b1-c7a3-422c-b97f-22218f7d3c75","Type":"ContainerStarted","Data":"fb705a6348a7749175927ff66789dbaa22dc7f745a780ff47e8c6435a6959db9"}
Jul 28 13:43:22.401842 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:22.401806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:43:23.063314 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.063288    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:23.085989 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.085961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7002-140696164038672_4473b4b1-c7a3-422c-b97f-22218f7d3c75/test-curl-7002-140696164038672/0.log"
Jul 28 13:43:23.113855 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.113823    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7002-140696164038672"]
Jul 28 13:43:23.118128 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.118100    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7002-140696164038672"]
Jul 28 13:43:23.208992 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.208912    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8j2\" (UniqueName: \"kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2\") pod \"4473b4b1-c7a3-422c-b97f-22218f7d3c75\" (UID: \"4473b4b1-c7a3-422c-b97f-22218f7d3c75\") "
Jul 28 13:43:23.211106 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.211072    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2" (OuterVolumeSpecName: "kube-api-access-ql8j2") pod "4473b4b1-c7a3-422c-b97f-22218f7d3c75" (UID: "4473b4b1-c7a3-422c-b97f-22218f7d3c75"). InnerVolumeSpecName "kube-api-access-ql8j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:23.242610 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.242570    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:23.243032 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.243014    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4473b4b1-c7a3-422c-b97f-22218f7d3c75" containerName="test-curl-7002-140696164038672"
Jul 28 13:43:23.243137 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.243034    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4473b4b1-c7a3-422c-b97f-22218f7d3c75" containerName="test-curl-7002-140696164038672"
Jul 28 13:43:23.243189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.243145    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4473b4b1-c7a3-422c-b97f-22218f7d3c75" containerName="test-curl-7002-140696164038672"
Jul 28 13:43:23.246257 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.246233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:23.253775 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.253727    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4473b4b1-c7a3-422c-b97f-22218f7d3c75" path="/var/lib/kubelet/pods/4473b4b1-c7a3-422c-b97f-22218f7d3c75/volumes"
Jul 28 13:43:23.254759 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.254697    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:23.310488 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.310447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h5c\" (UniqueName: \"kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c\") pod \"test-curl-7002-140696164039680\" (UID: \"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:23.310659 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.310499    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8j2\" (UniqueName: \"kubernetes.io/projected/4473b4b1-c7a3-422c-b97f-22218f7d3c75-kube-api-access-ql8j2\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:23.411462 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.411423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h5c\" (UniqueName: \"kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c\") pod \"test-curl-7002-140696164039680\" (UID: \"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:23.420286 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.420261    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h5c\" (UniqueName: \"kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c\") pod \"test-curl-7002-140696164039680\" (UID: \"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:23.557718 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.557689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:23.694384 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.694357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:23.697276 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:23.697246    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bd40aac_7cb2_41d1_962a_6883d0bc5f5f.slice/crio-29f0583775fb25d0e515c6d40ffd4e64e9a13c19f30d96f843523aae26feffcc WatchSource:0}: Error finding container 29f0583775fb25d0e515c6d40ffd4e64e9a13c19f30d96f843523aae26feffcc: Status 404 returned error can't find the container with id 29f0583775fb25d0e515c6d40ffd4e64e9a13c19f30d96f843523aae26feffcc
Jul 28 13:43:23.932369 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.932277    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164038672"
Jul 28 13:43:23.932369 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.932306    2562 scope.go:117] "RemoveContainer" containerID="db8cd4b47f070e47d5e3923329d030508ef7ecb43cc5c656a9504702298ef2bc"
Jul 28 13:43:23.933594 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:23.933550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164039680" event={"ID":"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f","Type":"ContainerStarted","Data":"29f0583775fb25d0e515c6d40ffd4e64e9a13c19f30d96f843523aae26feffcc"}
Jul 28 13:43:24.940706 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:24.940666    2562 generic.go:356] "Generic (PLEG): container finished" podID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" containerID="6c9231cf1f7514caaa0797376261d77513531f87e9e9135c5cbc9549a7574370" exitCode=0
Jul 28 13:43:24.941115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:24.940712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164039680" event={"ID":"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f","Type":"ContainerDied","Data":"6c9231cf1f7514caaa0797376261d77513531f87e9e9135c5cbc9549a7574370"}
Jul 28 13:43:25.741924 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:25.741872    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" podUID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" containerName="main" containerID="cri-o://d703895829035b44d8a02d94d19ba941c1b36599a741436185aa39248133341a" gracePeriod=45
Jul 28 13:43:25.947715 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:25.947683    2562 generic.go:356] "Generic (PLEG): container finished" podID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" containerID="d703895829035b44d8a02d94d19ba941c1b36599a741436185aa39248133341a" exitCode=0
Jul 28 13:43:25.948201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:25.947773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" event={"ID":"003cb8de-0fe7-4812-8e58-da4a8a4a434c","Type":"ContainerDied","Data":"d703895829035b44d8a02d94d19ba941c1b36599a741436185aa39248133341a"}
Jul 28 13:43:26.001021 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.000945    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:43:26.090160 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.090131    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.111588 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.111563    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7002-140696164039680_2bd40aac-7cb2-41d1-962a-6883d0bc5f5f/test-curl-7002-140696164039680/0.log"
Jul 28 13:43:26.139880 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.139852    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlx9g\" (UniqueName: \"kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.139910    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.139954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.139968    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.139990    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140051    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs\") pod \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\" (UID: \"003cb8de-0fe7-4812-8e58-da4a8a4a434c\") "
Jul 28 13:43:26.140328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:26.140328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache" (OuterVolumeSpecName: "model-cache") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:26.140328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140293    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home" (OuterVolumeSpecName: "home") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:26.140468 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140395    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.140468 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140412    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.140468 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140425    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.140686 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.140667    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:26.142242 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.142223    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm" (OuterVolumeSpecName: "dshm") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:26.142337 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.142284    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:43:26.142337 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.142297    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g" (OuterVolumeSpecName: "kube-api-access-dlx9g") pod "003cb8de-0fe7-4812-8e58-da4a8a4a434c" (UID: "003cb8de-0fe7-4812-8e58-da4a8a4a434c"). InnerVolumeSpecName "kube-api-access-dlx9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:26.147258 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.147235    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:26.240813 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.240784    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6h5c\" (UniqueName: \"kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c\") pod \"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f\" (UID: \"2bd40aac-7cb2-41d1-962a-6883d0bc5f5f\") "
Jul 28 13:43:26.241027 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.241016    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/003cb8de-0fe7-4812-8e58-da4a8a4a434c-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.241075 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.241030    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/003cb8de-0fe7-4812-8e58-da4a8a4a434c-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.241075 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.241039    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlx9g\" (UniqueName: \"kubernetes.io/projected/003cb8de-0fe7-4812-8e58-da4a8a4a434c-kube-api-access-dlx9g\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.242734 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.242703    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c" (OuterVolumeSpecName: "kube-api-access-f6h5c") pod "2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" (UID: "2bd40aac-7cb2-41d1-962a-6883d0bc5f5f"). InnerVolumeSpecName "kube-api-access-f6h5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:26.313402 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313326    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:26.313762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313729    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:26.313762 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313758    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:26.313852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313779    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" containerName="main"
Jul 28 13:43:26.313852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313786    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" containerName="main"
Jul 28 13:43:26.313852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313846    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:26.313944 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.313861    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" containerName="main"
Jul 28 13:43:26.317019 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.317001    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.329953 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.327797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:26.341937 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.341908    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6h5c\" (UniqueName: \"kubernetes.io/projected/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f-kube-api-access-f6h5c\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:26.442956 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.442918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6877\" (UniqueName: \"kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877\") pod \"test-curl-7002-140696164039680\" (UID: \"c4e58dcd-763f-414c-a3da-98f8383da5b8\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.543906 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.543865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6877\" (UniqueName: \"kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877\") pod \"test-curl-7002-140696164039680\" (UID: \"c4e58dcd-763f-414c-a3da-98f8383da5b8\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.553975 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.553951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6877\" (UniqueName: \"kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877\") pod \"test-curl-7002-140696164039680\" (UID: \"c4e58dcd-763f-414c-a3da-98f8383da5b8\") " pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.630700 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.630615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.638400 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.638362    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" podUID="638922aa-f562-4d15-85cf-1ee069ed2043" containerName="main" containerID="cri-o://1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d" gracePeriod=45
Jul 28 13:43:26.766985 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.766943    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:26.769910 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:26.769879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4e58dcd_763f_414c_a3da_98f8383da5b8.slice/crio-9c53aa529a477562422acbda98db2c74bc1eaf04c91b93ddc3d0c2617971f80f WatchSource:0}: Error finding container 9c53aa529a477562422acbda98db2c74bc1eaf04c91b93ddc3d0c2617971f80f: Status 404 returned error can't find the container with id 9c53aa529a477562422acbda98db2c74bc1eaf04c91b93ddc3d0c2617971f80f
Jul 28 13:43:26.894523 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.894467    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:43:26.953935 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.953902    2562 generic.go:356] "Generic (PLEG): container finished" podID="638922aa-f562-4d15-85cf-1ee069ed2043" containerID="1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d" exitCode=0
Jul 28 13:43:26.954394 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.953937    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"
Jul 28 13:43:26.954394 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.953977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" event={"ID":"638922aa-f562-4d15-85cf-1ee069ed2043","Type":"ContainerDied","Data":"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"}
Jul 28 13:43:26.954394 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.954015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk" event={"ID":"638922aa-f562-4d15-85cf-1ee069ed2043","Type":"ContainerDied","Data":"a1361a392c012c0c71f359b006fb091fa65e35531b9b1615d2d152232ef7da27"}
Jul 28 13:43:26.954394 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.954033    2562 scope.go:117] "RemoveContainer" containerID="1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"
Jul 28 13:43:26.955260 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.955242    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29f0583775fb25d0e515c6d40ffd4e64e9a13c19f30d96f843523aae26feffcc"
Jul 28 13:43:26.955355 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.955264    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:26.956640 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.956622    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"
Jul 28 13:43:26.956640 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.956637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc" event={"ID":"003cb8de-0fe7-4812-8e58-da4a8a4a434c","Type":"ContainerDied","Data":"2a7ca3503ce7af2e802f2d13733628f30762def5f449e650da8953272c603e12"}
Jul 28 13:43:26.958190 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.958172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164039680" event={"ID":"c4e58dcd-763f-414c-a3da-98f8383da5b8","Type":"ContainerStarted","Data":"9c53aa529a477562422acbda98db2c74bc1eaf04c91b93ddc3d0c2617971f80f"}
Jul 28 13:43:26.964482 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.964466    2562 scope.go:117] "RemoveContainer" containerID="1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"
Jul 28 13:43:26.964756 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:43:26.964723    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d\": container with ID starting with 1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d not found: ID does not exist" containerID="1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"
Jul 28 13:43:26.964808 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.964776    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d"} err="failed to get container status \"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d\": rpc error: code = NotFound desc = could not find container \"1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d\": container with ID starting with 1200f0dd902f703b52842ce921c7afa82239ae36546389a680d15245e31ab46d not found: ID does not exist"
Jul 28 13:43:26.964808 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.964793    2562 scope.go:117] "RemoveContainer" containerID="d703895829035b44d8a02d94d19ba941c1b36599a741436185aa39248133341a"
Jul 28 13:43:26.971385 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.971355    2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7002-140696164039680" oldPodUID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" podUID="c4e58dcd-763f-414c-a3da-98f8383da5b8"
Jul 28 13:43:26.986945 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.986923    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:43:26.988559 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:26.988540    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2c9e23/rate-test-model-2c9e23-kserve-c75974cbc-ftzrc"]
Jul 28 13:43:27.048630 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048611    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr6g8\" (UniqueName: \"kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.048729 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048653    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.048878 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.048914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048895    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.048965 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048952    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.049018 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.048986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir\") pod \"638922aa-f562-4d15-85cf-1ee069ed2043\" (UID: \"638922aa-f562-4d15-85cf-1ee069ed2043\") "
Jul 28 13:43:27.049137 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.049111    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache" (OuterVolumeSpecName: "model-cache") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:27.049381 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.049327    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.049447 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.049350    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:27.049564 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.049543    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home" (OuterVolumeSpecName: "home") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:27.051046 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.051022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm" (OuterVolumeSpecName: "dshm") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:43:27.051189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.051168    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:43:27.051189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.051178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8" (OuterVolumeSpecName: "kube-api-access-kr6g8") pod "638922aa-f562-4d15-85cf-1ee069ed2043" (UID: "638922aa-f562-4d15-85cf-1ee069ed2043"). InnerVolumeSpecName "kube-api-access-kr6g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:27.150820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.150766    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.150820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.150791    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638922aa-f562-4d15-85cf-1ee069ed2043-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.150820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.150799    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.150820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.150807    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr6g8\" (UniqueName: \"kubernetes.io/projected/638922aa-f562-4d15-85cf-1ee069ed2043-kube-api-access-kr6g8\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.150820 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.150815    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638922aa-f562-4d15-85cf-1ee069ed2043-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:27.252429 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.252397    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="003cb8de-0fe7-4812-8e58-da4a8a4a434c" path="/var/lib/kubelet/pods/003cb8de-0fe7-4812-8e58-da4a8a4a434c/volumes"
Jul 28 13:43:27.252705 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.252691    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd40aac-7cb2-41d1-962a-6883d0bc5f5f" path="/var/lib/kubelet/pods/2bd40aac-7cb2-41d1-962a-6883d0bc5f5f/volumes"
Jul 28 13:43:27.277342 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.277319    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:43:27.283430 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.283407    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8fd168/rate-test-model-8fd168-kserve-79cb78df95-pb4vk"]
Jul 28 13:43:27.964711 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.964620    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4e58dcd-763f-414c-a3da-98f8383da5b8" containerID="27f609a3591309b4331b74c71b47c9a8a7336378495013fc457170cc5b73d71c" exitCode=0
Jul 28 13:43:27.964711 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:27.964672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7002-140696164039680" event={"ID":"c4e58dcd-763f-414c-a3da-98f8383da5b8","Type":"ContainerDied","Data":"27f609a3591309b4331b74c71b47c9a8a7336378495013fc457170cc5b73d71c"}
Jul 28 13:43:29.105583 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.105553    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:29.130291 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.130254    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7002-140696164039680_c4e58dcd-763f-414c-a3da-98f8383da5b8/test-curl-7002-140696164039680/0.log"
Jul 28 13:43:29.155581 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.155542    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:29.159380 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.159356    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7002-140696164039680"]
Jul 28 13:43:29.253050 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.253012    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="638922aa-f562-4d15-85cf-1ee069ed2043" path="/var/lib/kubelet/pods/638922aa-f562-4d15-85cf-1ee069ed2043/volumes"
Jul 28 13:43:29.270143 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.270115    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6877\" (UniqueName: \"kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877\") pod \"c4e58dcd-763f-414c-a3da-98f8383da5b8\" (UID: \"c4e58dcd-763f-414c-a3da-98f8383da5b8\") "
Jul 28 13:43:29.272367 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.272343    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877" (OuterVolumeSpecName: "kube-api-access-t6877") pod "c4e58dcd-763f-414c-a3da-98f8383da5b8" (UID: "c4e58dcd-763f-414c-a3da-98f8383da5b8"). InnerVolumeSpecName "kube-api-access-t6877". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:43:29.371030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.370986    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6877\" (UniqueName: \"kubernetes.io/projected/c4e58dcd-763f-414c-a3da-98f8383da5b8-kube-api-access-t6877\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:43:29.976571 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.976544    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7002-140696164039680"
Jul 28 13:43:29.976787 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:29.976600    2562 scope.go:117] "RemoveContainer" containerID="27f609a3591309b4331b74c71b47c9a8a7336378495013fc457170cc5b73d71c"
Jul 28 13:43:30.018998 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.018966    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:43:30.019371 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019358    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="638922aa-f562-4d15-85cf-1ee069ed2043" containerName="main"
Jul 28 13:43:30.019424 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019375    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="638922aa-f562-4d15-85cf-1ee069ed2043" containerName="main"
Jul 28 13:43:30.019424 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019392    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e58dcd-763f-414c-a3da-98f8383da5b8" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:30.019424 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019398    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e58dcd-763f-414c-a3da-98f8383da5b8" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:30.019551 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019535    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e58dcd-763f-414c-a3da-98f8383da5b8" containerName="test-curl-7002-140696164039680"
Jul 28 13:43:30.019587 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.019560    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="638922aa-f562-4d15-85cf-1ee069ed2043" containerName="main"
Jul 28 13:43:30.024115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.024095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.027261 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.027240    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-15d223-openshift-default-dockercfg-gmjmj\""
Jul 28 13:43:30.036973 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.036949    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:43:30.178881 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.178881 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178930    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.178981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8b8z\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.179044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.179078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.179309 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.179139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280054 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280444 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280444 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280299    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8b8z\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280444 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280362    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280444 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280635 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280776 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.280838 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.281063 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.280823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.281121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.281027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.282635 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.282607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.282899 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.282880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.315483 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.315446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.315583 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.315489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8b8z\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z\") pod \"e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") " pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.334348 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.334319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:30.513651 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.513618    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:43:30.517114 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:30.517091    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod256186eb_4574_4041_9b49_8184906073a0.slice/crio-7205b4e7dc220eae06b201fdc2b36a375c826767416ca31303d6bee48adf21e8 WatchSource:0}: Error finding container 7205b4e7dc220eae06b201fdc2b36a375c826767416ca31303d6bee48adf21e8: Status 404 returned error can't find the container with id 7205b4e7dc220eae06b201fdc2b36a375c826767416ca31303d6bee48adf21e8
Jul 28 13:43:30.519343 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.519311    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:30.519437 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.519377    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:30.519437 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.519424    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:30.983455 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.983345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerStarted","Data":"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"}
Jul 28 13:43:30.983455 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:30.983407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerStarted","Data":"7205b4e7dc220eae06b201fdc2b36a375c826767416ca31303d6bee48adf21e8"}
Jul 28 13:43:31.008991 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.008932    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podStartSLOduration=2.008912509 podStartE2EDuration="2.008912509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:43:31.007386492 +0000 UTC m=+3124.324405485" watchObservedRunningTime="2026-07-28 13:43:31.008912509 +0000 UTC m=+3124.325931481"
Jul 28 13:43:31.251526 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.251497    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e58dcd-763f-414c-a3da-98f8383da5b8" path="/var/lib/kubelet/pods/c4e58dcd-763f-414c-a3da-98f8383da5b8/volumes"
Jul 28 13:43:31.335038 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.335007    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:31.339875 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.339850    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:31.990240 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.990207    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:31.991179 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:31.991158    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:40.128249 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:40.128211    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:43:48.031979 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:48.031882    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 13:43:48.031979 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:48.031958    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 13:43:49.074463 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.074425    2562 generic.go:356] "Generic (PLEG): container finished" podID="256186eb-4574-4041-9b49-8184906073a0" containerID="4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987" exitCode=0
Jul 28 13:43:49.074926 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.074501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerDied","Data":"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"}
Jul 28 13:43:49.074926 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.074891    2562 scope.go:117] "RemoveContainer" containerID="4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"
Jul 28 13:43:49.075912 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.075886    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:49.075969 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.075958    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:49.076012 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:49.075992    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:50.081475 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:50.081442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerStarted","Data":"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"}
Jul 28 13:43:50.335408 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:50.335315    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:50.335579 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:50.335557    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 13:43:50.335625 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:50.335608    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 13:43:51.335404 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:51.335366    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 13:43:51.335852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:51.335424    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 13:43:52.335712 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:52.335668    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 13:43:52.336150 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:52.335728    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 13:43:53.335371 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:53.335334    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 13:43:53.335579 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:53.335393    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 13:43:54.338988 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:54.338957    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:54.339375 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:54.339283    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:54.339865 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:54.339849    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:43:59.196025 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.195983    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:43:59.198721 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.198704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.204134 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.204110    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-220710-openshift-default-dockercfg-j9m7t\""
Jul 28 13:43:59.219267 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.219237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:43:59.248606 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248777 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248777 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248777 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5txp\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248854    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.248936 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.248911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349532 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349532 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349536    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349789 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5txp\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349789 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349789 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.349789 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350017 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350017 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350017 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.349932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350173 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.350029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350234 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.350199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350291 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.350244    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350413 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.350396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.350829 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.350807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.351958 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.351941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.352344 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.352326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.358614 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.358592    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.359024 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.359004    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5txp\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp\") pod \"e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") " pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.508912 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.508870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:43:59.653897 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.653866    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:43:59.661444 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:43:59.661405    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddcf0d1_afb1_4f8e_95b2_bd2337d4674f.slice/crio-dffd0c312d7d747737fa326656210d3b1fe4470e98257c96211d445e64105514 WatchSource:0}: Error finding container dffd0c312d7d747737fa326656210d3b1fe4470e98257c96211d445e64105514: Status 404 returned error can't find the container with id dffd0c312d7d747737fa326656210d3b1fe4470e98257c96211d445e64105514
Jul 28 13:43:59.663518 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.663486    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:59.663613 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.663550    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:43:59.663613 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:43:59.663581    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:44:00.136362 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:00.136323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerStarted","Data":"826336cfa82aeeac91ef7331a88e85d3d6713987a2d53171f6496de603886594"}
Jul 28 13:44:00.136544 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:00.136383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerStarted","Data":"dffd0c312d7d747737fa326656210d3b1fe4470e98257c96211d445e64105514"}
Jul 28 13:44:00.168376 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:00.168316    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podStartSLOduration=1.168294341 podStartE2EDuration="1.168294341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:44:00.165461845 +0000 UTC m=+3153.482480847" watchObservedRunningTime="2026-07-28 13:44:00.168294341 +0000 UTC m=+3153.485313379"
Jul 28 13:44:00.509232 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:00.509191    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:00.513878 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:00.513855    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:01.141509 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:01.141462    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:01.142429 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:01.142408    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:12.611622 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:12.611586    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:44:20.243473 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.243438    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerID="826336cfa82aeeac91ef7331a88e85d3d6713987a2d53171f6496de603886594" exitCode=0
Jul 28 13:44:20.243898 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.243520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerDied","Data":"826336cfa82aeeac91ef7331a88e85d3d6713987a2d53171f6496de603886594"}
Jul 28 13:44:20.243988 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.243973    2562 scope.go:117] "RemoveContainer" containerID="826336cfa82aeeac91ef7331a88e85d3d6713987a2d53171f6496de603886594"
Jul 28 13:44:20.245082 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.245044    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:44:20.245186 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.245170    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:44:20.245225 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:20.245212    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 28 13:44:21.252239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:21.252207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerStarted","Data":"43a653b85f6b663aacc9796bce0c741a11641719b7579d633c14f357af696f52"}
Jul 28 13:44:21.510897 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:21.510351    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:21.510897 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:21.510488    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 13:44:21.510897 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:21.510543    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 13:44:22.509616 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:22.509576    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 13:44:22.510244 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:22.510212    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 13:44:23.509677 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:23.509635    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 13:44:23.510321 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:23.510286    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 13:44:24.510237 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:24.510191    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 13:44:24.510819 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:24.510280    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 13:44:25.512943 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:25.512909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:25.513341 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:25.513172    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:44:25.513815 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:44:25.513797    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:45:00.169376 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.169345    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"]
Jul 28 13:45:00.171917 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.171897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"
Jul 28 13:45:00.175502 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.175464    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"]
Jul 28 13:45:00.176052 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.176024    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:45:00.177967 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.177950    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"]
Jul 28 13:45:00.178098 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.178080    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"
Jul 28 13:45:00.180175 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.180154    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"]
Jul 28 13:45:00.180301 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.180280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"
Jul 28 13:45:00.182489 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.182470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"
Jul 28 13:45:00.184836 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.184815    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"]
Jul 28 13:45:00.186692 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.186675    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"
Jul 28 13:45:00.195958 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.195936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"]
Jul 28 13:45:00.196415 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.196392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"
Jul 28 13:45:00.198515 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.198491    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"]
Jul 28 13:45:00.209781 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.209442    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"]
Jul 28 13:45:00.240377 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.240349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"
Jul 28 13:45:00.252998 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.252467    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"
Jul 28 13:45:00.386916 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.386862    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"]
Jul 28 13:45:00.388689 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:45:00.388659    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5595fc94_240a_42ad_9756_fafa47949d06.slice/crio-4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d WatchSource:0}: Error finding container 4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d: Status 404 returned error can't find the container with id 4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d
Jul 28 13:45:00.423135 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.423103    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"]
Jul 28 13:45:00.427089 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:45:00.427062    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf85d10_f0f2_4456_9f41_a882d8b2a205.slice/crio-e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26 WatchSource:0}: Error finding container e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26: Status 404 returned error can't find the container with id e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26
Jul 28 13:45:00.441790 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.441760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf" event={"ID":"5595fc94-240a-42ad-9756-fafa47949d06","Type":"ContainerStarted","Data":"4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d"}
Jul 28 13:45:00.443291 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.443261    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c" event={"ID":"6bf85d10-f0f2-4456-9f41-a882d8b2a205","Type":"ContainerStarted","Data":"e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26"}
Jul 28 13:45:00.447027 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.447003    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"]
Jul 28 13:45:00.463815 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:45:00.463786    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3c956c_8a19_4a48_ae98_0fa5d0b06efb.slice/crio-0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc WatchSource:0}: Error finding container 0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc: Status 404 returned error can't find the container with id 0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc
Jul 28 13:45:00.477500 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.477470    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"]
Jul 28 13:45:00.479864 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:45:00.479838    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8920c83a_b0b6_4ea5_85e6_15e19472c458.slice/crio-6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482 WatchSource:0}: Error finding container 6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482: Status 404 returned error can't find the container with id 6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482
Jul 28 13:45:00.901091 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.901051    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:45:00.904626 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.904605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:00.908353 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.908328    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-15d223\"/\"kube-root-ca.crt\""
Jul 28 13:45:00.908493 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.908472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-15d223\"/\"default-dockercfg-kg7g6\""
Jul 28 13:45:00.908493 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.908480    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-15d223\"/\"openshift-service-ca.crt\""
Jul 28 13:45:00.908585 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.908545    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-15d223\"/\"ipp-route-15d223-kserve-self-signed-certs\""
Jul 28 13:45:00.915445 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:00.915424    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:45:01.055696 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.055696 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.055955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7gl8\" (UniqueName: \"kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.055955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055806    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.055955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.055955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.055873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156437 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156598 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156598 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156598 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156779 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156779 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7gl8\" (UniqueName: \"kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.156925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.156904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.157049 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.157017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.157151 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.157069    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.158955 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.158935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.159201 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.159187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.166043 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.166022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7gl8\" (UniqueName: \"kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8\") pod \"ipp-route-15d223-kserve-7768d857c9-w4pjz\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") " pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.214963 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.214919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:01.373243 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.373222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:45:01.375174 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:45:01.375147    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6f3a4e4_eec9_427e_9ba9_5ca79d5ae5d2.slice/crio-fda50c5b617f531e858f67b161147486e17be1e2a3e02bbeed46d9c2fa89a55c WatchSource:0}: Error finding container fda50c5b617f531e858f67b161147486e17be1e2a3e02bbeed46d9c2fa89a55c: Status 404 returned error can't find the container with id fda50c5b617f531e858f67b161147486e17be1e2a3e02bbeed46d9c2fa89a55c
Jul 28 13:45:01.456504 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.456466    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf" event={"ID":"5595fc94-240a-42ad-9756-fafa47949d06","Type":"ContainerStarted","Data":"13cc96055d6fbfa78a68e8bebcc0752143edbad19f1f31813b3c80957d36eada"}
Jul 28 13:45:01.458066 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.458034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c" event={"ID":"8920c83a-b0b6-4ea5-85e6-15e19472c458","Type":"ContainerStarted","Data":"c279d7d20d7546181f9ca4ae73571fd3075eadfe2b63ab8494ab170720dd7c10"}
Jul 28 13:45:01.458220 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.458092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c" event={"ID":"8920c83a-b0b6-4ea5-85e6-15e19472c458","Type":"ContainerStarted","Data":"6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482"}
Jul 28 13:45:01.459476 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.459446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl" event={"ID":"0f3c956c-8a19-4a48-ae98-0fa5d0b06efb","Type":"ContainerStarted","Data":"b08008a81862c9b1ec7ee0d2d987a41c050c934883c0f6c497ffa1bceca20bf8"}
Jul 28 13:45:01.459581 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.459487    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl" event={"ID":"0f3c956c-8a19-4a48-ae98-0fa5d0b06efb","Type":"ContainerStarted","Data":"0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc"}
Jul 28 13:45:01.460584 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.460563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" event={"ID":"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2","Type":"ContainerStarted","Data":"fda50c5b617f531e858f67b161147486e17be1e2a3e02bbeed46d9c2fa89a55c"}
Jul 28 13:45:01.462072 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.462051    2562 generic.go:356] "Generic (PLEG): container finished" podID="6bf85d10-f0f2-4456-9f41-a882d8b2a205" containerID="6ee498e22877c8397f43e23a140f0601886c0cf96205cf863d18e6621d026907" exitCode=0
Jul 28 13:45:01.462163 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.462080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c" event={"ID":"6bf85d10-f0f2-4456-9f41-a882d8b2a205","Type":"ContainerDied","Data":"6ee498e22877c8397f43e23a140f0601886c0cf96205cf863d18e6621d026907"}
Jul 28 13:45:01.502122 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.502068    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf" podStartSLOduration=1.50204506 podStartE2EDuration="1.50204506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:45:01.49443086 +0000 UTC m=+3214.811449849" watchObservedRunningTime="2026-07-28 13:45:01.50204506 +0000 UTC m=+3214.819064081"
Jul 28 13:45:01.517166 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.517117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c" podStartSLOduration=1.5170963830000002 podStartE2EDuration="1.517096383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:45:01.514530868 +0000 UTC m=+3214.831549858" watchObservedRunningTime="2026-07-28 13:45:01.517096383 +0000 UTC m=+3214.834115374"
Jul 28 13:45:01.583339 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:01.583285    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl" podStartSLOduration=1.583269633 podStartE2EDuration="1.583269633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:45:01.581689834 +0000 UTC m=+3214.898708823" watchObservedRunningTime="2026-07-28 13:45:01.583269633 +0000 UTC m=+3214.900288622"
Jul 28 13:45:02.469359 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.469267    2562 generic.go:356] "Generic (PLEG): container finished" podID="5595fc94-240a-42ad-9756-fafa47949d06" containerID="13cc96055d6fbfa78a68e8bebcc0752143edbad19f1f31813b3c80957d36eada" exitCode=0
Jul 28 13:45:02.469359 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.469335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf" event={"ID":"5595fc94-240a-42ad-9756-fafa47949d06","Type":"ContainerDied","Data":"13cc96055d6fbfa78a68e8bebcc0752143edbad19f1f31813b3c80957d36eada"}
Jul 28 13:45:02.470837 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.470801    2562 generic.go:356] "Generic (PLEG): container finished" podID="8920c83a-b0b6-4ea5-85e6-15e19472c458" containerID="c279d7d20d7546181f9ca4ae73571fd3075eadfe2b63ab8494ab170720dd7c10" exitCode=0
Jul 28 13:45:02.470953 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.470882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c" event={"ID":"8920c83a-b0b6-4ea5-85e6-15e19472c458","Type":"ContainerDied","Data":"c279d7d20d7546181f9ca4ae73571fd3075eadfe2b63ab8494ab170720dd7c10"}
Jul 28 13:45:02.472314 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.472292    2562 generic.go:356] "Generic (PLEG): container finished" podID="0f3c956c-8a19-4a48-ae98-0fa5d0b06efb" containerID="b08008a81862c9b1ec7ee0d2d987a41c050c934883c0f6c497ffa1bceca20bf8" exitCode=0
Jul 28 13:45:02.472407 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.472373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl" event={"ID":"0f3c956c-8a19-4a48-ae98-0fa5d0b06efb","Type":"ContainerDied","Data":"b08008a81862c9b1ec7ee0d2d987a41c050c934883c0f6c497ffa1bceca20bf8"}
Jul 28 13:45:02.473922 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.473901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" event={"ID":"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2","Type":"ContainerStarted","Data":"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"}
Jul 28 13:45:02.571818 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.571752    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" podStartSLOduration=2.571720033 podStartE2EDuration="2.571720033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:45:02.571197193 +0000 UTC m=+3215.888216183" watchObservedRunningTime="2026-07-28 13:45:02.571720033 +0000 UTC m=+3215.888739047"
Jul 28 13:45:02.609290 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:02.609268    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"
Jul 28 13:45:03.480644 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.480599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c" event={"ID":"6bf85d10-f0f2-4456-9f41-a882d8b2a205","Type":"ContainerDied","Data":"e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26"}
Jul 28 13:45:03.481123 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.480664    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1dbddf983b144cab41432395a2bad9afaaf2c2099e2e615af325ef9085a7f26"
Jul 28 13:45:03.481123 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.481082    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"
Jul 28 13:45:03.632899 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.632873    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"
Jul 28 13:45:03.691271 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.691241    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"
Jul 28 13:45:03.694515 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:03.694493    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"
Jul 28 13:45:04.487300 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.487271    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf"
Jul 28 13:45:04.487784 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.487278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754105-qgcvf" event={"ID":"5595fc94-240a-42ad-9756-fafa47949d06","Type":"ContainerDied","Data":"4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d"}
Jul 28 13:45:04.487784 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.487428    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4603ca72143b39f954bc247e43d39de649e3072e37c1766b6b93901c740b522d"
Jul 28 13:45:04.488575 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.488557    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"
Jul 28 13:45:04.488673 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.488581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c" event={"ID":"8920c83a-b0b6-4ea5-85e6-15e19472c458","Type":"ContainerDied","Data":"6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482"}
Jul 28 13:45:04.488673 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.488621    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bbfebe16dec42a8a7490c0b732dcbe026575a519872ed8c069bdc7d91074482"
Jul 28 13:45:04.489852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.489830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl" event={"ID":"0f3c956c-8a19-4a48-ae98-0fa5d0b06efb","Type":"ContainerDied","Data":"0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc"}
Jul 28 13:45:04.489968 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.489853    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"
Jul 28 13:45:04.489968 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.489865    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bdd1e39611db2c0dc384a47bb11722403b40a86fc3a18f942c1e3976fac7ccc"
Jul 28 13:45:04.610817 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:04.610784    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:45:11.215328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:11.215290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:11.215732 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:11.215353    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:11.227886 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:11.227859    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:11.541346 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:11.541316    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:45:35.107239 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:45:35.107155    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:46:01.537215 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:01.537181    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:46:17.567022 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.566981    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"]
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567408    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8920c83a-b0b6-4ea5-85e6-15e19472c458" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567420    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8920c83a-b0b6-4ea5-85e6-15e19472c458" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567437    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5595fc94-240a-42ad-9756-fafa47949d06" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567442    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5595fc94-240a-42ad-9756-fafa47949d06" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567448    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f3c956c-8a19-4a48-ae98-0fa5d0b06efb" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567454    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f3c956c-8a19-4a48-ae98-0fa5d0b06efb" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567461    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf85d10-f0f2-4456-9f41-a882d8b2a205" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567466    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf85d10-f0f2-4456-9f41-a882d8b2a205" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567539    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5595fc94-240a-42ad-9756-fafa47949d06" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567550    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f3c956c-8a19-4a48-ae98-0fa5d0b06efb" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567561    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf85d10-f0f2-4456-9f41-a882d8b2a205" containerName="cleanup"
Jul 28 13:46:17.567637 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.567570    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8920c83a-b0b6-4ea5-85e6-15e19472c458" containerName="cleanup"
Jul 28 13:46:17.570665 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.570644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:17.574083 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.574058    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:46:17.574216 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.574127    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:46:17.580852 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.580822    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"]
Jul 28 13:46:17.691878 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.691841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7\" (UID: \"011bd62e-f27c-41eb-af51-0b8694b9b0d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:17.792478 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.792443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7\" (UID: \"011bd62e-f27c-41eb-af51-0b8694b9b0d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:17.793139 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.793118    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7\" (UID: \"011bd62e-f27c-41eb-af51-0b8694b9b0d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:17.883273 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:17.883191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:18.023524 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:18.023495    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"]
Jul 28 13:46:18.025873 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:46:18.025839    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011bd62e_f27c_41eb_af51_0b8694b9b0d1.slice/crio-4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940 WatchSource:0}: Error finding container 4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940: Status 404 returned error can't find the container with id 4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940
Jul 28 13:46:18.884110 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:18.884071    2562 generic.go:356] "Generic (PLEG): container finished" podID="011bd62e-f27c-41eb-af51-0b8694b9b0d1" containerID="14fca90c2413c1b4524b0353c033f398d1a0d71577047bef7e406b8b6e7e63ab" exitCode=0
Jul 28 13:46:18.884489 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:18.884132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7" event={"ID":"011bd62e-f27c-41eb-af51-0b8694b9b0d1","Type":"ContainerDied","Data":"14fca90c2413c1b4524b0353c033f398d1a0d71577047bef7e406b8b6e7e63ab"}
Jul 28 13:46:18.884489 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:18.884198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7" event={"ID":"011bd62e-f27c-41eb-af51-0b8694b9b0d1","Type":"ContainerStarted","Data":"4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940"}
Jul 28 13:46:20.024692 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.024667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:20.223679 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.223593    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca\") pod \"011bd62e-f27c-41eb-af51-0b8694b9b0d1\" (UID: \"011bd62e-f27c-41eb-af51-0b8694b9b0d1\") "
Jul 28 13:46:20.223994 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.223967    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "011bd62e-f27c-41eb-af51-0b8694b9b0d1" (UID: "011bd62e-f27c-41eb-af51-0b8694b9b0d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:46:20.224068 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.224056    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/011bd62e-f27c-41eb-af51-0b8694b9b0d1-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:46:20.896487 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.896456    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"
Jul 28 13:46:20.896670 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.896456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7" event={"ID":"011bd62e-f27c-41eb-af51-0b8694b9b0d1","Type":"ContainerDied","Data":"4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940"}
Jul 28 13:46:20.896670 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:20.896582    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4da9a5c7e94fb275cee7976b451a113d5b749b9a05a249f9e740c8bfb1a04940"
Jul 28 13:46:22.783306 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:22.783235    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"]
Jul 28 13:46:22.785839 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:22.785812    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ed36e3-g85k7"]
Jul 28 13:46:23.251920 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:23.251883    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="011bd62e-f27c-41eb-af51-0b8694b9b0d1" path="/var/lib/kubelet/pods/011bd62e-f27c-41eb-af51-0b8694b9b0d1/volumes"
Jul 28 13:46:26.812156 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:26.812113    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:46:55.150777 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:46:55.150677    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:47:01.881384 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:01.881346    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:47:06.344475 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:06.344440    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:47:16.901467 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:16.901418    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" podUID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" containerName="main" containerID="cri-o://e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445" gracePeriod=45
Jul 28 13:47:17.156597 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.156535    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:47:17.196763 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.196710    2562 generic.go:356] "Generic (PLEG): container finished" podID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" containerID="e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445" exitCode=0
Jul 28 13:47:17.196934 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.196775    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"
Jul 28 13:47:17.196934 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.196778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" event={"ID":"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2","Type":"ContainerDied","Data":"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"}
Jul 28 13:47:17.196934 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.196820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz" event={"ID":"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2","Type":"ContainerDied","Data":"fda50c5b617f531e858f67b161147486e17be1e2a3e02bbeed46d9c2fa89a55c"}
Jul 28 13:47:17.196934 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.196837    2562 scope.go:117] "RemoveContainer" containerID="e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"
Jul 28 13:47:17.208543 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.208521    2562 scope.go:117] "RemoveContainer" containerID="e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"
Jul 28 13:47:17.208973 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:47:17.208951    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445\": container with ID starting with e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445 not found: ID does not exist" containerID="e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"
Jul 28 13:47:17.209044 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.208982    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445"} err="failed to get container status \"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445\": rpc error: code = NotFound desc = could not find container \"e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445\": container with ID starting with e1b36d6f0aa000b99c38e9321dac16f6d6b366526253914a76c4369df3f35445 not found: ID does not exist"
Jul 28 13:47:17.273900 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.273869    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7gl8\" (UniqueName: \"kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274073 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.273935    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274073 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274018    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274195 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274086    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274195 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274195 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274159    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs\") pod \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\" (UID: \"c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2\") "
Jul 28 13:47:17.274355 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274224    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:17.274355 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274273    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:17.274468 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home" (OuterVolumeSpecName: "home") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:17.274530 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274488    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-model-cache\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.274530 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274502    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tmp-dir\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.274530 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.274510    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-home\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.276250 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.276223    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm" (OuterVolumeSpecName: "dshm") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:17.276355 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.276268    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8" (OuterVolumeSpecName: "kube-api-access-p7gl8") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "kube-api-access-p7gl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:47:17.276445 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.276425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" (UID: "c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:47:17.375179 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.375141    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-dshm\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.375179 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.375169    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-tls-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.375179 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.375180    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7gl8\" (UniqueName: \"kubernetes.io/projected/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2-kube-api-access-p7gl8\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:17.522021 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.521988    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:47:17.528235 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:17.528205    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-15d223/ipp-route-15d223-kserve-7768d857c9-w4pjz"]
Jul 28 13:47:19.251865 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:19.251835    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" path="/var/lib/kubelet/pods/c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2/volumes"
Jul 28 13:47:28.287353 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.287313    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:47:28.287818 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.287608    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy" containerID="cri-o://94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a" gracePeriod=30
Jul 28 13:47:28.680065 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.679979    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"]
Jul 28 13:47:28.680386 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680373    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" containerName="main"
Jul 28 13:47:28.680430 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680387    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" containerName="main"
Jul 28 13:47:28.680430 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680409    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011bd62e-f27c-41eb-af51-0b8694b9b0d1" containerName="revoke-keys"
Jul 28 13:47:28.680430 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680415    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="011bd62e-f27c-41eb-af51-0b8694b9b0d1" containerName="revoke-keys"
Jul 28 13:47:28.680528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680469    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="011bd62e-f27c-41eb-af51-0b8694b9b0d1" containerName="revoke-keys"
Jul 28 13:47:28.680528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.680479    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6f3a4e4-eec9-427e-9ba9-5ca79d5ae5d2" containerName="main"
Jul 28 13:47:28.684910 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.684890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:28.687686 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.687665    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:47:28.687832 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.687664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:47:28.693501 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.693466    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"]
Jul 28 13:47:28.785556 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.785515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9\" (UID: \"a9a8ec2e-62c4-426b-b41c-e64bda9d3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:28.886313 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.886274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9\" (UID: \"a9a8ec2e-62c4-426b-b41c-e64bda9d3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:28.886980 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.886959    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9\" (UID: \"a9a8ec2e-62c4-426b-b41c-e64bda9d3665\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:28.996173 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:28.996147    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:29.132005 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:29.131979    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"]
Jul 28 13:47:29.134239 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:47:29.134213    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a8ec2e_62c4_426b_b41c_e64bda9d3665.slice/crio-f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9 WatchSource:0}: Error finding container f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9: Status 404 returned error can't find the container with id f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9
Jul 28 13:47:29.259965 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:29.259885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9" event={"ID":"a9a8ec2e-62c4-426b-b41c-e64bda9d3665","Type":"ContainerStarted","Data":"f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9"}
Jul 28 13:47:30.266134 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:30.266096    2562 generic.go:356] "Generic (PLEG): container finished" podID="a9a8ec2e-62c4-426b-b41c-e64bda9d3665" containerID="0b976f4d43a580bdcd2e24d0c113421be12b94dcb7f5176c6d7334535d7dedd4" exitCode=0
Jul 28 13:47:30.266559 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:30.266187    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9" event={"ID":"a9a8ec2e-62c4-426b-b41c-e64bda9d3665","Type":"ContainerDied","Data":"0b976f4d43a580bdcd2e24d0c113421be12b94dcb7f5176c6d7334535d7dedd4"}
Jul 28 13:47:31.402037 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:31.402012    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:31.511238 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:31.511204    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca\") pod \"a9a8ec2e-62c4-426b-b41c-e64bda9d3665\" (UID: \"a9a8ec2e-62c4-426b-b41c-e64bda9d3665\") "
Jul 28 13:47:31.511598 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:31.511569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9a8ec2e-62c4-426b-b41c-e64bda9d3665" (UID: "a9a8ec2e-62c4-426b-b41c-e64bda9d3665"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:47:31.612086 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:31.612005    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9a8ec2e-62c4-426b-b41c-e64bda9d3665-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:32.276682 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:32.276652    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"
Jul 28 13:47:32.276682 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:32.276676    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9" event={"ID":"a9a8ec2e-62c4-426b-b41c-e64bda9d3665","Type":"ContainerDied","Data":"f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9"}
Jul 28 13:47:32.276921 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:32.276726    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0a260842c581840091c51788084202da8c898633d9c71620a6ea5e9708f55f9"
Jul 28 13:47:33.541593 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.541568    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:47:33.731507 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731408    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731675 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731512    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731675 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731544    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731675 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731581    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731675 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8b8z\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731677    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731713    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731769    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:33.731984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731786    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.731984 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731885    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy\") pod \"256186eb-4574-4041-9b49-8184906073a0\" (UID: \"256186eb-4574-4041-9b49-8184906073a0\") "
Jul 28 13:47:33.732233 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.731999    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:33.732286 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.732263    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.732333 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.732283    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-credential-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.732772 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.732716    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:33.732881 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.732853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:47:33.733189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.733163    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:33.734964 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.734930    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:33.735335 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.735305    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:47:33.736050 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.736027    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z" (OuterVolumeSpecName: "kube-api-access-q8b8z") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "kube-api-access-q8b8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:47:33.736189 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.736103    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "256186eb-4574-4041-9b49-8184906073a0" (UID: "256186eb-4574-4041-9b49-8184906073a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:47:33.780034 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.779992    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"]
Jul 28 13:47:33.786192 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.786164    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-220710-29754105-twk9c"]
Jul 28 13:47:33.833922 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833882    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-istio-token\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.833922 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833923    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256186eb-4574-4041-9b49-8184906073a0-istio-podinfo\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.834121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833937    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-data\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.834121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833952    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8b8z\" (UniqueName: \"kubernetes.io/projected/256186eb-4574-4041-9b49-8184906073a0-kube-api-access-q8b8z\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.834121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833967    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-workload-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.834121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833979    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256186eb-4574-4041-9b49-8184906073a0-istiod-ca-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.834121 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.833992    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256186eb-4574-4041-9b49-8184906073a0-istio-envoy\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:33.986435 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.986342    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"]
Jul 28 13:47:33.990214 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:33.990189    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-220710-sr6m9"]
Jul 28 13:47:34.287880 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.287847    2562 generic.go:356] "Generic (PLEG): container finished" podID="256186eb-4574-4041-9b49-8184906073a0" containerID="94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a" exitCode=0
Jul 28 13:47:34.288030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.287903    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"
Jul 28 13:47:34.288030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.287925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerDied","Data":"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"}
Jul 28 13:47:34.288030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.287982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6" event={"ID":"256186eb-4574-4041-9b49-8184906073a0","Type":"ContainerDied","Data":"7205b4e7dc220eae06b201fdc2b36a375c826767416ca31303d6bee48adf21e8"}
Jul 28 13:47:34.288030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.288005    2562 scope.go:117] "RemoveContainer" containerID="94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"
Jul 28 13:47:34.298079 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.298057    2562 scope.go:117] "RemoveContainer" containerID="4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"
Jul 28 13:47:34.306806 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.306787    2562 scope.go:117] "RemoveContainer" containerID="94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"
Jul 28 13:47:34.307087 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:47:34.307067    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a\": container with ID starting with 94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a not found: ID does not exist" containerID="94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"
Jul 28 13:47:34.307130 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.307097    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a"} err="failed to get container status \"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a\": rpc error: code = NotFound desc = could not find container \"94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a\": container with ID starting with 94f47c79a256ad029e437c2761c0284470ce5f374d442b18f451d9b87b55d63a not found: ID does not exist"
Jul 28 13:47:34.307130 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.307118    2562 scope.go:117] "RemoveContainer" containerID="4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"
Jul 28 13:47:34.307380 ip-10-0-132-196 kubenswrapper[2562]: E0728 13:47:34.307364    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987\": container with ID starting with 4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987 not found: ID does not exist" containerID="4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"
Jul 28 13:47:34.307417 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.307387    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987"} err="failed to get container status \"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987\": rpc error: code = NotFound desc = could not find container \"4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987\": container with ID starting with 4e0ee4eb85adbd88a5a75f65ed964fb1ddb9d2b7a9ab3e3d0ea3a75068b7f987 not found: ID does not exist"
Jul 28 13:47:34.312644 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.312621    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:47:34.316946 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:34.316924    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-15d223-openshift-default-8dbc8b6f6-n4qc6"]
Jul 28 13:47:35.251377 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:35.251340    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="256186eb-4574-4041-9b49-8184906073a0" path="/var/lib/kubelet/pods/256186eb-4574-4041-9b49-8184906073a0/volumes"
Jul 28 13:47:35.251785 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:35.251626    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf85d10-f0f2-4456-9f41-a882d8b2a205" path="/var/lib/kubelet/pods/6bf85d10-f0f2-4456-9f41-a882d8b2a205/volumes"
Jul 28 13:47:35.251905 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:35.251889    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9a8ec2e-62c4-426b-b41c-e64bda9d3665" path="/var/lib/kubelet/pods/a9a8ec2e-62c4-426b-b41c-e64bda9d3665/volumes"
Jul 28 13:47:36.897826 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:36.897787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:47:41.219375 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:41.219282    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:47:41.219790 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:41.219563    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy" containerID="cri-o://43a653b85f6b663aacc9796bce0c741a11641719b7579d633c14f357af696f52" gracePeriod=30
Jul 28 13:47:46.354808 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.354767    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerID="43a653b85f6b663aacc9796bce0c741a11641719b7579d633c14f357af696f52" exitCode=0
Jul 28 13:47:46.355269 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.354828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerDied","Data":"43a653b85f6b663aacc9796bce0c741a11641719b7579d633c14f357af696f52"}
Jul 28 13:47:46.355269 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.354892    2562 scope.go:117] "RemoveContainer" containerID="826336cfa82aeeac91ef7331a88e85d3d6713987a2d53171f6496de603886594"
Jul 28 13:47:46.460634 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.460608    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:47:46.556238 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556209    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556238 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556240    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556458 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556458 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556458 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556374    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556608 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:46.556608 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556608 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556591    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556795 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556641    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5txp\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556795 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556683    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:46.556795 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy\") pod \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\" (UID: \"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f\") "
Jul 28 13:47:46.556795 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556723    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:46.556795 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.556775    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:47:46.557071 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.557058    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-credential-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.557118 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.557079    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.557118 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.557092    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-workload-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.557118 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.557105    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istiod-ca-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.557789 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.557730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:46.559059 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.559024    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp" (OuterVolumeSpecName: "kube-api-access-c5txp") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "kube-api-access-c5txp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:47:46.559172 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.559122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:47:46.559172 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.559125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:47:46.559172 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.559121    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" (UID: "0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:47:46.658586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.658549    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-data\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.658586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.658579    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5txp\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-kube-api-access-c5txp\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.658586 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.658594    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-envoy\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.658858 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.658604    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-token\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:46.658858 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:46.658611    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f-istio-podinfo\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:47:47.360972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:47.360945    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"
Jul 28 13:47:47.360972 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:47.360958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn" event={"ID":"0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f","Type":"ContainerDied","Data":"dffd0c312d7d747737fa326656210d3b1fe4470e98257c96211d445e64105514"}
Jul 28 13:47:47.361464 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:47.361016    2562 scope.go:117] "RemoveContainer" containerID="43a653b85f6b663aacc9796bce0c741a11641719b7579d633c14f357af696f52"
Jul 28 13:47:47.385426 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:47.385395    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:47:47.387313 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:47.387289    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-220710-openshift-default-55f9b9d584-czppn"]
Jul 28 13:47:49.251280 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:49.251241    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" path="/var/lib/kubelet/pods/0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f/volumes"
Jul 28 13:47:52.369698 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:52.369658    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"]
Jul 28 13:47:52.373220 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:52.373191    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2c9e23-29754105-qj6fl"]
Jul 28 13:47:53.251528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:53.251493    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f3c956c-8a19-4a48-ae98-0fa5d0b06efb" path="/var/lib/kubelet/pods/0f3c956c-8a19-4a48-ae98-0fa5d0b06efb/volumes"
Jul 28 13:47:55.816194 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:47:55.816154    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:48:00.118454 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118420    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"]
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118802    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118814    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118822    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118828    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118841    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118846    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118866    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a8ec2e-62c4-426b-b41c-e64bda9d3665" containerName="revoke-keys"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118874    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a8ec2e-62c4-426b-b41c-e64bda9d3665" containerName="revoke-keys"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118886    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:00.118889 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118894    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:00.119198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118947    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:00.119198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118954    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9a8ec2e-62c4-426b-b41c-e64bda9d3665" containerName="revoke-keys"
Jul 28 13:48:00.119198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118963    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.119198 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.118972    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="256186eb-4574-4041-9b49-8184906073a0" containerName="istio-proxy"
Jul 28 13:48:00.123506 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.123488    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:00.126365 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.126345    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 13:48:00.126490 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.126475    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6qfh\""
Jul 28 13:48:00.135290 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.135260    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"]
Jul 28 13:48:00.185030 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.184994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g\" (UID: \"bfe34a76-61fe-41e6-937e-802d1ecf7918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:00.286050 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.286016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g\" (UID: \"bfe34a76-61fe-41e6-937e-802d1ecf7918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:00.286628 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.286608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g\" (UID: \"bfe34a76-61fe-41e6-937e-802d1ecf7918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:00.434546 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.434463    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:00.586022 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:00.585997    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"]
Jul 28 13:48:00.588079 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:48:00.588047    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe34a76_61fe_41e6_937e_802d1ecf7918.slice/crio-57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463 WatchSource:0}: Error finding container 57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463: Status 404 returned error can't find the container with id 57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463
Jul 28 13:48:01.439517 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:01.439484    2562 generic.go:356] "Generic (PLEG): container finished" podID="bfe34a76-61fe-41e6-937e-802d1ecf7918" containerID="9163bc348ef39f592430993f8ffeb6bba947119a18e24b8d60d96930f516cfa3" exitCode=0
Jul 28 13:48:01.439925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:01.439569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g" event={"ID":"bfe34a76-61fe-41e6-937e-802d1ecf7918","Type":"ContainerDied","Data":"9163bc348ef39f592430993f8ffeb6bba947119a18e24b8d60d96930f516cfa3"}
Jul 28 13:48:01.439925 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:01.439630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g" event={"ID":"bfe34a76-61fe-41e6-937e-802d1ecf7918","Type":"ContainerStarted","Data":"57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463"}
Jul 28 13:48:02.571758 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:02.571719    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:02.710782 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:02.710666    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca\") pod \"bfe34a76-61fe-41e6-937e-802d1ecf7918\" (UID: \"bfe34a76-61fe-41e6-937e-802d1ecf7918\") "
Jul 28 13:48:02.711096 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:02.711070    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bfe34a76-61fe-41e6-937e-802d1ecf7918" (UID: "bfe34a76-61fe-41e6-937e-802d1ecf7918"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:48:02.812320 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:02.812282    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe34a76-61fe-41e6-937e-802d1ecf7918-maas-api-service-ca\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:03.450021 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:03.449978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g" event={"ID":"bfe34a76-61fe-41e6-937e-802d1ecf7918","Type":"ContainerDied","Data":"57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463"}
Jul 28 13:48:03.450191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:03.450034    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57f42bed28efaf57c062e196777413ba7a89ec0c8811bab754867b6f67c75463"
Jul 28 13:48:03.450191 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:03.449996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"
Jul 28 13:48:10.724898 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:10.724865    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"]
Jul 28 13:48:10.726910 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:10.726886    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8fd168-29754105-w7n6c"]
Jul 28 13:48:10.949603 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:10.949565    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"]
Jul 28 13:48:10.952144 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:10.952117    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8fd168-qkj8g"]
Jul 28 13:48:11.251546 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:11.251514    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8920c83a-b0b6-4ea5-85e6-15e19472c458" path="/var/lib/kubelet/pods/8920c83a-b0b6-4ea5-85e6-15e19472c458/volumes"
Jul 28 13:48:11.251847 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:11.251833    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfe34a76-61fe-41e6-937e-802d1ecf7918" path="/var/lib/kubelet/pods/bfe34a76-61fe-41e6-937e-802d1ecf7918/volumes"
Jul 28 13:48:13.756012 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:13.755971    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-72hhp"]
Jul 28 13:48:18.361289 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:18.361194    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:48:18.361800 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:18.361448    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" containerID="cri-o://d6e5d685f6224ebb335f37fff3160f3f20bb0bc9911f63371e9f07a1593527b0" gracePeriod=30
Jul 28 13:48:18.968828 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:18.968786    2562 patch_prober.go:28] interesting pod/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 13:48:18.968991 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:18.968853    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:48:23.554459 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.554423    2562 generic.go:356] "Generic (PLEG): container finished" podID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerID="d6e5d685f6224ebb335f37fff3160f3f20bb0bc9911f63371e9f07a1593527b0" exitCode=0
Jul 28 13:48:23.554880 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.554504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerDied","Data":"d6e5d685f6224ebb335f37fff3160f3f20bb0bc9911f63371e9f07a1593527b0"}
Jul 28 13:48:23.554880 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.554572    2562 scope.go:117] "RemoveContainer" containerID="36f87724229bf31aee8fc3eeefe859336a7599bcbf79f9184f557bb55832a068"
Jul 28 13:48:23.616159 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.616128    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:48:23.707537 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707450    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707537 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707537 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707514    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707555    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707599    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707683    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbw8l\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l\") pod \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\" (UID: \"243e9e2d-a329-41e2-9ab9-f64184f2b5aa\") "
Jul 28 13:48:23.707848 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.707804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:48:23.708181 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.708002    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-credential-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.708181 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.708021    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:48:23.708181 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.708119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:48:23.708358 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.708338    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:48:23.708836 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.708815    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:48:23.710089 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.710065    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:48:23.710227 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.710204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l" (OuterVolumeSpecName: "kube-api-access-vbw8l") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "kube-api-access-vbw8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:48:23.710491 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.710471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:48:23.710491 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.710477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "243e9e2d-a329-41e2-9ab9-f64184f2b5aa" (UID: "243e9e2d-a329-41e2-9ab9-f64184f2b5aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:48:23.808894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808860    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-certs\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.808894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808889    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbw8l\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-kube-api-access-vbw8l\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.808894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808899    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-podinfo\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.809129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808910    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-workload-socket\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.809129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808920    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istiod-ca-cert\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.809129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808928    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-envoy\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.809129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808935    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-token\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:23.809129 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:23.808942    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243e9e2d-a329-41e2-9ab9-f64184f2b5aa-istio-data\") on node \"ip-10-0-132-196.ec2.internal\" DevicePath \"\""
Jul 28 13:48:24.561610 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:24.561573    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"
Jul 28 13:48:24.561610 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:24.561593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6" event={"ID":"243e9e2d-a329-41e2-9ab9-f64184f2b5aa","Type":"ContainerDied","Data":"9b6a239c6fc1886bda1f31c90253735e99808cb7dd0fe61d89697f9b76713458"}
Jul 28 13:48:24.562153 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:24.561660    2562 scope.go:117] "RemoveContainer" containerID="d6e5d685f6224ebb335f37fff3160f3f20bb0bc9911f63371e9f07a1593527b0"
Jul 28 13:48:24.587481 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:24.587453    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:48:24.593002 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:24.592976    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd168-openshift-default-86bcd94c84-kf8b6"]
Jul 28 13:48:25.251965 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:25.251928    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" path="/var/lib/kubelet/pods/243e9e2d-a329-41e2-9ab9-f64184f2b5aa/volumes"
Jul 28 13:48:26.903511 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:26.903474    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bb544cf8-8sjpf_85cdd0cd-74d2-4e4c-926b-f86352a7b123/manager/0.log"
Jul 28 13:48:26.911078 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:26.911056    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bb544cf8-8sjpf_85cdd0cd-74d2-4e4c-926b-f86352a7b123/copy-manifests/0.log"
Jul 28 13:48:27.880768 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:27.880662    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754105-qgcvf_5595fc94-240a-42ad-9756-fafa47949d06/cleanup/0.log"
Jul 28 13:48:29.493374 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:29.493343    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g5crp_8607ad9c-3b18-4197-a774-152e6be53fd1/manager/0.log"
Jul 28 13:48:30.119327 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:30.119297    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-72hhp_81f13ddb-42d0-4d33-8e13-52f17557f3e6/limitador/0.log"
Jul 28 13:48:31.005356 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:31.005328    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7897958f89-twvr2_ef38dd5b-65d2-4606-83cf-5931a7823eaf/kube-auth-proxy/0.log"
Jul 28 13:48:32.805920 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:32.805886    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lkcsf_8006dbcb-9735-4495-abea-75bcfbbe43cc/main/0.log"
Jul 28 13:48:40.425754 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.425697    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4gt2h/must-gather-56q2x"]
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426064    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426076    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426093    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe34a76-61fe-41e6-937e-802d1ecf7918" containerName="revoke-keys"
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426100    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe34a76-61fe-41e6-937e-802d1ecf7918" containerName="revoke-keys"
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426110    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.426138 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426116    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.426328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426166    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ddcf0d1-afb1-4f8e-95b2-bd2337d4674f" containerName="istio-proxy"
Jul 28 13:48:40.426328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426176    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe34a76-61fe-41e6-937e-802d1ecf7918" containerName="revoke-keys"
Jul 28 13:48:40.426328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426187    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.426328 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.426192    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="243e9e2d-a329-41e2-9ab9-f64184f2b5aa" containerName="istio-proxy"
Jul 28 13:48:40.429286 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.429270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.432231 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.432210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4gt2h\"/\"kube-root-ca.crt\""
Jul 28 13:48:40.433127 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.433101    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4gt2h\"/\"openshift-service-ca.crt\""
Jul 28 13:48:40.433127 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.433122    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4gt2h\"/\"default-dockercfg-tljvg\""
Jul 28 13:48:40.441888 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.441867    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4gt2h/must-gather-56q2x"]
Jul 28 13:48:40.560267 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.560229    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz26n\" (UniqueName: \"kubernetes.io/projected/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-kube-api-access-xz26n\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.560462 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.560372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-must-gather-output\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.661646 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.661606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz26n\" (UniqueName: \"kubernetes.io/projected/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-kube-api-access-xz26n\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.661866 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.661725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-must-gather-output\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.662031 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.662011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-must-gather-output\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.671185 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.671157    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz26n\" (UniqueName: \"kubernetes.io/projected/fb891b7c-84ed-4f49-9e78-4b5097f4fcff-kube-api-access-xz26n\") pod \"must-gather-56q2x\" (UID: \"fb891b7c-84ed-4f49-9e78-4b5097f4fcff\") " pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.738735 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.738658    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4gt2h/must-gather-56q2x"
Jul 28 13:48:40.878157 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.878128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4gt2h/must-gather-56q2x"]
Jul 28 13:48:40.880000 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:48:40.879963    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb891b7c_84ed_4f49_9e78_4b5097f4fcff.slice/crio-3d2284f900414b608f3e49a0b51a2c84383caf4b95d2c0b30495b08f824f7e9f WatchSource:0}: Error finding container 3d2284f900414b608f3e49a0b51a2c84383caf4b95d2c0b30495b08f824f7e9f: Status 404 returned error can't find the container with id 3d2284f900414b608f3e49a0b51a2c84383caf4b95d2c0b30495b08f824f7e9f
Jul 28 13:48:40.881906 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:40.881887    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:48:41.650498 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:41.650462    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4gt2h/must-gather-56q2x" event={"ID":"fb891b7c-84ed-4f49-9e78-4b5097f4fcff","Type":"ContainerStarted","Data":"3d2284f900414b608f3e49a0b51a2c84383caf4b95d2c0b30495b08f824f7e9f"}
Jul 28 13:48:42.657556 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:42.657515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4gt2h/must-gather-56q2x" event={"ID":"fb891b7c-84ed-4f49-9e78-4b5097f4fcff","Type":"ContainerStarted","Data":"0c6c3aa111b1c3409b198a117f74c05792dc1c7fda51df4bfcb981f86bcc90b9"}
Jul 28 13:48:42.658006 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:42.657571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4gt2h/must-gather-56q2x" event={"ID":"fb891b7c-84ed-4f49-9e78-4b5097f4fcff","Type":"ContainerStarted","Data":"d001b39e1cd5a27c0cb52d0fc175d5cdf7df00503944a34f9c414168a0bd7fae"}
Jul 28 13:48:42.691504 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:42.691440    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4gt2h/must-gather-56q2x" podStartSLOduration=1.993253217 podStartE2EDuration="2.691420101s" totalImagesPullingTime="698.166884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:42.690308476 +0000 UTC m=+3436.007327467" watchObservedRunningTime="2026-07-28 13:48:42.691420101 +0000 UTC m=+3436.008439091"
Jul 28 13:48:43.434894 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:43.434866    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fv5kv_fe4d3851-5860-4c58-9921-741f4a99e615/global-pull-secret-syncer/0.log"
Jul 28 13:48:43.685907 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:43.685822    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mfdq5_4deb847e-8bcb-47ba-869f-2f4bde506df0/konnectivity-agent/0.log"
Jul 28 13:48:43.712697 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:43.712666    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-196.ec2.internal_1e60603cd4d0f2a54399ab7713287926/haproxy/0.log"
Jul 28 13:48:48.976329 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:48.976274    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g5crp_8607ad9c-3b18-4197-a774-152e6be53fd1/manager/0.log"
Jul 28 13:48:49.344548 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:49.344513    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-72hhp_81f13ddb-42d0-4d33-8e13-52f17557f3e6/limitador/0.log"
Jul 28 13:48:51.179580 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.179547    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/alertmanager/0.log"
Jul 28 13:48:51.205415 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.205392    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/config-reloader/0.log"
Jul 28 13:48:51.239510 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.239426    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/kube-rbac-proxy-web/0.log"
Jul 28 13:48:51.267238 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.267206    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/kube-rbac-proxy/0.log"
Jul 28 13:48:51.292115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.292088    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/kube-rbac-proxy-metric/0.log"
Jul 28 13:48:51.317696 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.317665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/prom-label-proxy/0.log"
Jul 28 13:48:51.361023 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.360980    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae13f4cd-3879-416c-b8db-cd679a36d87c/init-config-reloader/0.log"
Jul 28 13:48:51.440886 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.440850    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-d6s52_d70a1860-7e59-4c14-819e-c3519bc6a589/kube-state-metrics/0.log"
Jul 28 13:48:51.466317 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.466290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-d6s52_d70a1860-7e59-4c14-819e-c3519bc6a589/kube-rbac-proxy-main/0.log"
Jul 28 13:48:51.498421 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.498389    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-d6s52_d70a1860-7e59-4c14-819e-c3519bc6a589/kube-rbac-proxy-self/0.log"
Jul 28 13:48:51.531269 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.531237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c8c8b8466-2ggg2_3eab24ab-2682-4b75-93af-1707c2bb7eaa/metrics-server/0.log"
Jul 28 13:48:51.567073 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.567010    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-gs855_12fc22ca-dc5f-43b0-b64f-59a609f23463/monitoring-plugin/0.log"
Jul 28 13:48:51.826136 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.826025    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7dgsl_7c152ab7-1110-4f3e-8498-3b887d27fb64/node-exporter/0.log"
Jul 28 13:48:51.849715 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.849680    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7dgsl_7c152ab7-1110-4f3e-8498-3b887d27fb64/kube-rbac-proxy/0.log"
Jul 28 13:48:51.874655 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.874624    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7dgsl_7c152ab7-1110-4f3e-8498-3b887d27fb64/init-textfile/0.log"
Jul 28 13:48:51.907194 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.907171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mzz4b_b6a3186d-d911-4511-b390-bf1e454dc0ba/kube-rbac-proxy-main/0.log"
Jul 28 13:48:51.934929 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.934844    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mzz4b_b6a3186d-d911-4511-b390-bf1e454dc0ba/kube-rbac-proxy-self/0.log"
Jul 28 13:48:51.962474 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:51.962436    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mzz4b_b6a3186d-d911-4511-b390-bf1e454dc0ba/openshift-state-metrics/0.log"
Jul 28 13:48:52.006675 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.006641    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/prometheus/0.log"
Jul 28 13:48:52.032021 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.031991    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/config-reloader/0.log"
Jul 28 13:48:52.058409 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.058381    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/thanos-sidecar/0.log"
Jul 28 13:48:52.083313 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.083232    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/kube-rbac-proxy-web/0.log"
Jul 28 13:48:52.109212 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.109179    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/kube-rbac-proxy/0.log"
Jul 28 13:48:52.161962 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.161929    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/kube-rbac-proxy-thanos/0.log"
Jul 28 13:48:52.212016 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.211989    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba756922-f0ee-4333-b34d-bff9ed99faed/init-config-reloader/0.log"
Jul 28 13:48:52.277001 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.276974    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-kqs6f_64b5f305-e14b-4ac9-baff-ce97e9b5ba56/prometheus-operator/0.log"
Jul 28 13:48:52.303036 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.303002    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-kqs6f_64b5f305-e14b-4ac9-baff-ce97e9b5ba56/kube-rbac-proxy/0.log"
Jul 28 13:48:52.333273 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.333244    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-2lzkk_25b7d641-ff4a-439e-926c-83c3a75a6e4c/prometheus-operator-admission-webhook/0.log"
Jul 28 13:48:52.473056 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.472922    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/thanos-query/0.log"
Jul 28 13:48:52.497717 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.497684    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/kube-rbac-proxy-web/0.log"
Jul 28 13:48:52.520127 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.520098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/kube-rbac-proxy/0.log"
Jul 28 13:48:52.541956 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.541929    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/prom-label-proxy/0.log"
Jul 28 13:48:52.564534 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.564490    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/kube-rbac-proxy-rules/0.log"
Jul 28 13:48:52.593160 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:52.593131    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5865f4ccb8-vg5r4_e00e332d-7e3b-4005-9d61-3fc1dab4432d/kube-rbac-proxy-metrics/0.log"
Jul 28 13:48:53.375604 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.375565    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"]
Jul 28 13:48:53.388769 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.388715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.393975 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.393892    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"]
Jul 28 13:48:53.491327 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.491266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-proc\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.491528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.491376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-sys\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.491528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.491416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-podres\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.491528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.491458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-lib-modules\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.491528 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.491495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvn9g\" (UniqueName: \"kubernetes.io/projected/2d5fe940-2b9f-432b-8c3c-3f98636777e1-kube-api-access-lvn9g\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592160 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-proc\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-sys\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-podres\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-lib-modules\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592351 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvn9g\" (UniqueName: \"kubernetes.io/projected/2d5fe940-2b9f-432b-8c3c-3f98636777e1-kube-api-access-lvn9g\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592870 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-podres\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592870 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-lib-modules\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.592870 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.592843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-proc\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.593177 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.593158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d5fe940-2b9f-432b-8c3c-3f98636777e1-sys\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.603055 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.603023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvn9g\" (UniqueName: \"kubernetes.io/projected/2d5fe940-2b9f-432b-8c3c-3f98636777e1-kube-api-access-lvn9g\") pod \"perf-node-gather-daemonset-xstqw\" (UID: \"2d5fe940-2b9f-432b-8c3c-3f98636777e1\") " pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.704240 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.704107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:53.910904 ip-10-0-132-196 kubenswrapper[2562]: W0728 13:48:53.910846    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d5fe940_2b9f_432b_8c3c_3f98636777e1.slice/crio-da3d9f72f3ae077f42d12156caf57ba9bbf4c8e7250f2c94341d56d4ae05e6a5 WatchSource:0}: Error finding container da3d9f72f3ae077f42d12156caf57ba9bbf4c8e7250f2c94341d56d4ae05e6a5: Status 404 returned error can't find the container with id da3d9f72f3ae077f42d12156caf57ba9bbf4c8e7250f2c94341d56d4ae05e6a5
Jul 28 13:48:53.912258 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:53.912065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"]
Jul 28 13:48:54.748993 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:54.748954    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw" event={"ID":"2d5fe940-2b9f-432b-8c3c-3f98636777e1","Type":"ContainerStarted","Data":"e560a763fa14ea3a5c373fd93c8f5afca02626494847f39488b155658627a3c5"}
Jul 28 13:48:54.749484 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:54.749020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw" event={"ID":"2d5fe940-2b9f-432b-8c3c-3f98636777e1","Type":"ContainerStarted","Data":"da3d9f72f3ae077f42d12156caf57ba9bbf4c8e7250f2c94341d56d4ae05e6a5"}
Jul 28 13:48:54.749807 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:54.749785    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:48:54.769008 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:54.768949    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw" podStartSLOduration=1.768934489 podStartE2EDuration="1.768934489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:54.767291867 +0000 UTC m=+3448.084310857" watchObservedRunningTime="2026-07-28 13:48:54.768934489 +0000 UTC m=+3448.085953478"
Jul 28 13:48:56.809226 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:56.809143    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g2pq_f68cd3a9-d5e2-46a2-95e1-fa630b6ab563/dns/0.log"
Jul 28 13:48:56.832406 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:56.832380    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g2pq_f68cd3a9-d5e2-46a2-95e1-fa630b6ab563/kube-rbac-proxy/0.log"
Jul 28 13:48:56.974486 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:56.974442    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jslj5_87a22f14-9114-4f72-a467-40611a128b83/dns-node-resolver/0.log"
Jul 28 13:48:57.673699 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:57.673655    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dkmbx_d85a6d0b-63f6-4e44-869f-7043450252d6/node-ca/0.log"
Jul 28 13:48:58.927814 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:58.927780    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7897958f89-twvr2_ef38dd5b-65d2-4606-83cf-5931a7823eaf/kube-auth-proxy/0.log"
Jul 28 13:48:59.937187 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:48:59.937153    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pwgs_4b0560dc-5cae-4942-90cd-d11f2e0de254/serve-healthcheck-canary/0.log"
Jul 28 13:49:00.621098 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:00.621070    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mplhd_32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e/kube-rbac-proxy/0.log"
Jul 28 13:49:00.643115 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:00.643081    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mplhd_32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e/exporter/0.log"
Jul 28 13:49:00.670126 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:00.670075    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mplhd_32b5b4f4-aea6-40b6-b8ce-8d99e755bc0e/extractor/0.log"
Jul 28 13:49:01.773454 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:01.773426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4gt2h/perf-node-gather-daemonset-xstqw"
Jul 28 13:49:03.828948 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:03.828919    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7897958f89-twvr2_ef38dd5b-65d2-4606-83cf-5931a7823eaf/kube-auth-proxy/0.log"
Jul 28 13:49:03.865304 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:03.865277    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bb544cf8-8sjpf_85cdd0cd-74d2-4e4c-926b-f86352a7b123/manager/0.log"
Jul 28 13:49:03.894914 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:03.894357    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bb544cf8-8sjpf_85cdd0cd-74d2-4e4c-926b-f86352a7b123/copy-manifests/0.log"
Jul 28 13:49:05.420711 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:05.420675    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5fcd6f4d74-ct49f_d54f5603-b633-45f5-94c2-a99080eed723/manager/0.log"
Jul 28 13:49:05.699997 ip-10-0-132-196 kubenswrapper[2562]: I0728 13:49:05.699908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7897958f89-twvr2_ef38dd5b-65d2-4606-83cf-5931a7823eaf/kube-auth-proxy/0.log"