Jul 03 14:11:37.450464 ip-10-0-133-223 systemd[1]: Starting Kubernetes Kubelet...
Jul 03 14:11:37.917788 ip-10-0-133-223 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:11:37.917788 ip-10-0-133-223 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 03 14:11:37.917788 ip-10-0-133-223 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:11:37.917788 ip-10-0-133-223 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 03 14:11:37.917788 ip-10-0-133-223 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:11:37.918516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.917841    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 03 14:11:37.922773 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922755    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.922773 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922772    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922779    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922783    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922786    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922790    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922793    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922796    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922798    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922801    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922804    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922807    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922810    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922812    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922815    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922817    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922820    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922822    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922825    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922832    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922835    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.922844 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922837    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922840    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922842    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922845    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922848    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922850    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922854    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922856    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922859    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922861    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922864    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922866    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922870    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922874    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922876    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922879    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922882    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922884    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922887    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922889    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.923336 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922892    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922894    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922897    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922900    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922903    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922905    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922907    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922910    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922912    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922915    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922917    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922920    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922923    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922925    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922930    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922933    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922936    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922938    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922941    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.923816 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922943    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922946    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922949    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922951    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922954    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922956    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922959    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922961    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922964    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922966    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922969    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922971    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922973    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922976    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922978    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922981    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922983    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922985    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922988    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922990    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.924305 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922993    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922995    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.922998    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923004    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923007    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923010    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923013    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923016    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923018    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923021    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923023    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923026    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923028    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923031    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923033    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923035    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923038    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923040    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923043    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923763    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.924811 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923771    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923775    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923778    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923780    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923783    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923786    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923789    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923792    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923794    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923797    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923800    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923802    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923805    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923808    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923810    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923813    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923816    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923819    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923822    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923825    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.925296 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923827    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923830    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923833    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923836    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923838    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923841    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923843    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923846    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923848    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923851    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923854    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923856    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923859    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923861    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923864    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923867    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923869    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923872    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923874    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923876    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.925779 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923879    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923881    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923885    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923888    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923890    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923894    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923898    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923901    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923903    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923906    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923908    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923911    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923914    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923918    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923920    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923923    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923925    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923928    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923930    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.926277 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923932    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923936    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923940    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923943    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923946    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923949    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923951    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923954    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923957    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923960    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923964    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923966    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923969    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923972    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923975    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923977    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923980    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923983    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923985    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923988    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.926740 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923991    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923993    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923996    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.923999    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924002    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924005    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924008    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924010    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924013    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924015    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924018    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924020    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924023    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924025    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924028    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924030    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924033    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924035    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924037    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924436    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.927242 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924442    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924445    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924447    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924450    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924453    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924456    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924460    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924464    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924467    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924470    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924473    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924476    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924479    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924482    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924485    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924488    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924491    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924494    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924497    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.927723 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924499    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924501    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924504    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924507    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924509    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924512    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924514    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924517    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924520    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924524    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924526    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924529    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924532    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924534    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924537    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924541    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924545    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924548    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924550    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.928223 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924554    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924557    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924559    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924562    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924564    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924567    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924569    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924572    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924574    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924577    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924580    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924582    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924973    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924976    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924979    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924981    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924984    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924987    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924989    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924991    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924994    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.928686 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924996    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.924998    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925001    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925003    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925006    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925008    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925011    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925013    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925015    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925018    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925020    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925023    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925026    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925028    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925031    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925033    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925036    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925038    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925040    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.929201 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925043    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925045    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925047    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925049    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925053    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925056    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925058    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925061    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925063    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925065    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925068    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925070    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925073    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925075    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925077    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925079    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925082    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925085    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925087    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925090    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.929695 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925162    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925170    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925176    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925194    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925198    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925201    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925205    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925209    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925212    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925215    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925220    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925223    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925226    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925229    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925232    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925235    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925237    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925240    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925247    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925250    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925253    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925256    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925259    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925262    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 03 14:11:37.930171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925265    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925268    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925272    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925274    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925280    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925283    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925286    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925288    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925293    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925295    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925299    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925302    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925305    2561 flags.go:64] FLAG: --enable-server="true"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925308    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925312    2561 flags.go:64] FLAG: --event-burst="100"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925315    2561 flags.go:64] FLAG: --event-qps="50"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925318    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925321    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925323    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925327    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925330    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925333    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925336    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925339    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925342    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 03 14:11:37.930753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925345    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925347    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925350    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925353    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925356    2561 flags.go:64] FLAG: --feature-gates=""
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925359    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925362    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925365    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925368    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925371    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925374    2561 flags.go:64] FLAG: --help="false"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925378    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925381    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925384    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925387    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925390    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925393    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925396    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925402    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925404    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925407    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925410    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925413    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925415    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 03 14:11:37.931515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925418    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925421    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925424    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925427    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925430    2561 flags.go:64] FLAG: --lock-file=""
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925432    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925435    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925438    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925443    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925446    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925448    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925452    2561 flags.go:64] FLAG: --logging-format="text"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925454    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925458    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925460    2561 flags.go:64] FLAG: --manifest-url=""
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925463    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925467    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925470    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925474    2561 flags.go:64] FLAG: --max-pods="110"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925481    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925484    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925486    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925489    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925492    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925495    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 03 14:11:37.932092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925498    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925506    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925508    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925511    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925515    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925517    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925523    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925526    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925529    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925532    2561 flags.go:64] FLAG: --port="10250"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925535    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925537    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04c8261f38c6a0117"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925540    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925543    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925546    2561 flags.go:64] FLAG: --register-node="true"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925549    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925552    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925555    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925558    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925561    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925564    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925567    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925570    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925572    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925575    2561 flags.go:64] FLAG: --runonce="false"
Jul 03 14:11:37.932769 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925578    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925583    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925586    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925588    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925591    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925594    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925597    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925600    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925603    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925607    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925610    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925613    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925615    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925618    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925621    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925626    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925629    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925632    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925635    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925638    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925640    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925643    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925646    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925649    2561 flags.go:64] FLAG: --v="2"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925652    2561 flags.go:64] FLAG: --version="false"
Jul 03 14:11:37.933637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925661    2561 flags.go:64] FLAG: --vmodule=""
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925664    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.925668    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925779    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925783    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925786    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925789    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925791    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925795    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925797    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925800    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925803    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925805    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925808    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925810    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925812    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925816    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925819    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925821    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925824    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.934513 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925826    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925829    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925832    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925834    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925836    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925839    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925841    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925844    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925846    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925848    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925851    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925853    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925856    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925858    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925861    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925864    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925866    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925869    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925871    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925873    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.935012 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925877    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925881    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925885    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925888    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925892    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925895    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925898    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925900    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925905    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925907    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925910    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925913    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925916    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925920    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925922    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925925    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925928    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925930    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.935510 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925932    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925935    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925937    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925940    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925942    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925944    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925947    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925950    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925952    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925955    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925958    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925960    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925963    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925965    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925969    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925971    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925974    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925976    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925978    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925981    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.935946 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925983    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925986    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925990    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925992    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925995    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.925997    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926000    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926002    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926005    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926007    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926010    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926012    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926015    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926017    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926020    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926022    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926024    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926027    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926029    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926032    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.936543 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926034    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926036    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926039    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.926042    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.926849    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.935270    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 03 14:11:37.937024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.935285    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.937042    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.937051    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937111    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937117    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937121    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937124    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937126    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937130    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937133    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937135    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937138    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937141    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937143    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937146    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937148    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937151    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937153    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937156    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.937192 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937158    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937161    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937164    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937167    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937169    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937172    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937174    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937188    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937191    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937194    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937196    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937199    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937202    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937204    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937207    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937209    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937212    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937214    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937217    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937219    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.937656 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937222    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937225    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937230    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937232    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937235    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937237    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937240    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937242    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937245    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937248    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937250    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937252    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937255    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937257    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937261    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937263    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937266    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937268    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937271    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937273    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.938127 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937276    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937279    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937281    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937284    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937287    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937289    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937291    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937294    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937296    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937299    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937301    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937304    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937306    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937308    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937311    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937314    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937316    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937319    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937321    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937323    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.938611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937326    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937328    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937330    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937338    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937341    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937344    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937348    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937353    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937356    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937359    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937362    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937365    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937367    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937370    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937373    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937376    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937378    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937381    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937383    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.939126 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937385    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937388    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937390    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937393    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.937397    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937511    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937517    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937520    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937523    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937526    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937529    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937532    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937535    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937538    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:11:37.939608 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937540    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937543    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937546    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937549    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937551    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937555    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937557    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937560    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937563    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937566    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937569    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937571    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937574    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937576    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937578    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937581    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937583    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937586    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937588    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937591    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:11:37.939952 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937593    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937596    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937598    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937601    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937603    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937605    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937608    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937610    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937612    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937615    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937617    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937619    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937622    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937624    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937627    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937629    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937633    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937636    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937638    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937641    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:11:37.940455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937643    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937646    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937648    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937650    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937653    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937655    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937658    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937660    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937663    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937665    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937668    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937671    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937673    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937675    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937678    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937680    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937683    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937685    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937688    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937690    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:11:37.940924 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937692    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937695    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937697    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937700    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937702    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937705    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937707    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937710    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937712    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937716    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937718    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937721    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937723    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937726    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937730    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937733    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937736    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937739    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937742    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:11:37.941410 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937744    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937747    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937749    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937752    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937754    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937757    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937759    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937761    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937764    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937766    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:37.937769    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.937773    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.937901    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 03 14:11:37.941868 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.940629    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 03 14:11:37.942391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.942379    2561 server.go:1019] "Starting client certificate rotation"
Jul 03 14:11:37.942499 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.942481    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 14:11:37.942528 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.942519    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 14:11:37.947255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.947163    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 14:11:37.948304 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.948291    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 14:11:37.964731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.964714    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 03 14:11:37.966788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.966766    2561 log.go:25] "Validated CRI v1 image API"
Jul 03 14:11:37.967716 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.967698    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 03 14:11:37.972346 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.972322    2561 fs.go:135] Filesystem UUIDs: map[44ce6402-1276-4584-963c-b34bc5dcaa69:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9d086240-9787-42f7-a9f5-edb47c30b345:/dev/nvme0n1p3]
Jul 03 14:11:37.972346 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.972340    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 03 14:11:37.973470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.973453    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 14:11:37.979279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.979159    2561 manager.go:216] Machine: {Timestamp:2026-07-03 14:11:37.977713458 +0000 UTC m=+0.391609900 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098871 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2214853f966624d4781f278ecdd40d SystemUUID:ec221485-3f96-6624-d478-1f278ecdd40d BootID:b28c44ae-9c64-4484-8427-43631d44b081 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:90:63:1c:a5:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:90:63:1c:a5:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:d4:96:bd:94:b8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 03 14:11:37.979279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.979271    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 03 14:11:37.979402 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.979348    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 03 14:11:37.982939 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.982917    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 03 14:11:37.983086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.982943    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 03 14:11:37.983719 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.983710    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 03 14:11:37.983748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.983722    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 03 14:11:37.983748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.983730    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 14:11:37.984380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.984370    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 14:11:37.985697 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.985688    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 03 14:11:37.989036 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989025    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 14:11:37.989154 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989145    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 03 14:11:37.989210 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989175    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 03 14:11:37.989484 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989474    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 03 14:11:37.989534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989515    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 03 14:11:37.989534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989525    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 03 14:11:37.989534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.989534    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 03 14:11:37.990833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.990814    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 14:11:37.990912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.990842    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 14:11:37.996565 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.996544    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-25mdk"
Jul 03 14:11:37.997862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.997840    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 03 14:11:37.998588 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:37.998569    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 03 14:11:37.998641 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:37.998582    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 03 14:11:37.999932 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.999918    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 03 14:11:37.999979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:37.999949    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 03 14:11:38.001018 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001003    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 03 14:11:38.001018 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001022    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001031    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001041    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001053    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001058    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001066    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001073    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001081    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001086    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001094    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 03 14:11:38.001121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001099    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 03 14:11:38.001961 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001952    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 03 14:11:38.001961 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.001961    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 03 14:11:38.003987 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.003969    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 03 14:11:38.005057 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.005045    2561 server.go:1268] "Started kubelet"
Jul 03 14:11:38.005117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.005101    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-25mdk"
Jul 03 14:11:38.005228 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.005159    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 03 14:11:38.005314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.005287    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 03 14:11:38.005448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.005299    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 03 14:11:38.005934 ip-10-0-133-223 systemd[1]: Started Kubernetes Kubelet.
Jul 03 14:11:38.007797 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.007777    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 03 14:11:38.008136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.008123    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 03 14:11:38.012128 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.012109    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 03 14:11:38.013471 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.013453    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 03 14:11:38.013471 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.013461    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 03 14:11:38.013620 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.013550    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.014164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014123    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 03 14:11:38.014164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014129    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 03 14:11:38.014164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014165    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 03 14:11:38.014356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014308    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 03 14:11:38.014356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014318    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 03 14:11:38.014442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014369    2561 factory.go:55] Registering systemd factory
Jul 03 14:11:38.014442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014382    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 03 14:11:38.014699 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014688    2561 factory.go:153] Registering CRI-O factory
Jul 03 14:11:38.014737 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014701    2561 factory.go:223] Registration of the crio container factory successfully
Jul 03 14:11:38.014772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014743    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 03 14:11:38.014772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014764    2561 factory.go:103] Registering Raw factory
Jul 03 14:11:38.014831 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.014774    2561 manager.go:1196] Started watching for new ooms in manager
Jul 03 14:11:38.015383 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.015367    2561 manager.go:349] Starting recovery of all containers
Jul 03 14:11:38.022500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.022475    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:38.023340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.023308    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 03 14:11:38.027062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.026961    2561 manager.go:354] Recovery completed
Jul 03 14:11:38.028070 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.028051    2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.031699 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.031685    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.033402 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033381    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.033476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033421    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.033476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033435    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.033962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033949    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 03 14:11:38.033962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033956    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.034074 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033962    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 03 14:11:38.034074 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.033983    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 14:11:38.034656 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.034639    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.034750 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.034668    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.034750 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.034678    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.035158 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.035147    2561 policy_none.go:49] "None policy: Start"
Jul 03 14:11:38.035208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.035162    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 03 14:11:38.035208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.035172    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 03 14:11:38.036169 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.036158    2561 policy_none.go:47] "Start"
Jul 03 14:11:38.069281 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069267    2561 manager.go:337] "Starting Device Plugin manager"
Jul 03 14:11:38.069353 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.069294    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 03 14:11:38.069353 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069304    2561 server.go:87] "Starting device plugin registration server"
Jul 03 14:11:38.069544 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069533    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 03 14:11:38.069583 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069550    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 03 14:11:38.069636 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069623    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 03 14:11:38.069754 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069696    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 03 14:11:38.069754 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.069706    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 03 14:11:38.070250 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.070232    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 03 14:11:38.070302 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.070286    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.170452 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.170382    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.171855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.171321    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.171855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.171349    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.171855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.171359    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.171855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.171384    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.181114 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.181093    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.181248 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.181117    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-223.ec2.internal\": node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.184217 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.184196    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 03 14:11:38.184290 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.184232    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 03 14:11:38.184290 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.184251    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 03 14:11:38.184290 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.184279    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 03 14:11:38.186699 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.186684    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:38.229579 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.229558    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.284667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.284639    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"]
Jul 03 14:11:38.284740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.284723    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.285667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.285651    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.285766 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.285688    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.285766 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.285704    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.287050 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287034    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.287162 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287147    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.287230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287192    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.287740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287714    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.287740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287728    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.287850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287754    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.287850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287758    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.287850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287766    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.287850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.287769    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.289002 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.288988    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.289079 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.289013    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:11:38.289650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.289636    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:11:38.289742 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.289655    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:11:38.289742 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.289663    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:11:38.308711 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.308688    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.312641 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.312626    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.315803 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.315785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e1adb89e96ffa1aea4eb2057cc3c519-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"0e1adb89e96ffa1aea4eb2057cc3c519\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.315877 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.315817    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.315877 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.315840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.330165 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.330147    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.416788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e1adb89e96ffa1aea4eb2057cc3c519-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"0e1adb89e96ffa1aea4eb2057cc3c519\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.416936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416788    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.416936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.416936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416851    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e1adb89e96ffa1aea4eb2057cc3c519-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"0e1adb89e96ffa1aea4eb2057cc3c519\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.416936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416863    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.416936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.416880    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/954d8049aaecb57133d945db60ce4b63-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"954d8049aaecb57133d945db60ce4b63\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.430303 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.430249    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.531357 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.531327    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.612582 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.612533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.616574 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.616323    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:38.632375 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.632354    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.732914 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.732842    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.833369 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.833349    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.933825 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:38.933808    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found"
Jul 03 14:11:38.941969 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.941952    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 03 14:11:38.942097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.942081    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 03 14:11:38.942136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.942108    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 03 14:11:38.996908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:38.996855    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:39.009520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.009490    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 14:06:38 +0000 UTC" deadline="2027-12-22 00:19:42.48297229 +0000 UTC"
Jul 03 14:11:39.009520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.009514    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12874h8m3.473460778s"
Jul 03 14:11:39.013587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.013572    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 03 14:11:39.014038 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.014024    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:39.024640 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.024621    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 14:11:39.026122 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.026108    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 14:11:39.028323 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.028311    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"
Jul 03 14:11:39.038534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.038515    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 14:11:39.053135 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.053115    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcktj"
Jul 03 14:11:39.060166 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.060150    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcktj"
Jul 03 14:11:39.377493 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:39.377453    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1adb89e96ffa1aea4eb2057cc3c519.slice/crio-25e9e516e8f5e10df56933c25f38bbcb9dc85f49d38a999967d550cf401240da WatchSource:0}: Error finding container 25e9e516e8f5e10df56933c25f38bbcb9dc85f49d38a999967d550cf401240da: Status 404 returned error can't find the container with id 25e9e516e8f5e10df56933c25f38bbcb9dc85f49d38a999967d550cf401240da
Jul 03 14:11:39.377774 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:39.377754    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954d8049aaecb57133d945db60ce4b63.slice/crio-cc163467ec2ec58d7d8c34c01e2f48d01695c47d804b4843c690957def741204 WatchSource:0}: Error finding container cc163467ec2ec58d7d8c34c01e2f48d01695c47d804b4843c690957def741204: Status 404 returned error can't find the container with id cc163467ec2ec58d7d8c34c01e2f48d01695c47d804b4843c690957def741204
Jul 03 14:11:39.383586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.383570    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:11:39.566203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.566101    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:39.991452 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:39.991422    2561 apiserver.go:52] "Watching apiserver"
Jul 03 14:11:40.001743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.001712    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 03 14:11:40.002343 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.002312    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-gnpsb","openshift-network-operator/iptables-alerter-q585w","openshift-ovn-kubernetes/ovnkube-node-7rddn","kube-system/konnectivity-agent-clz2s","openshift-dns/node-resolver-778zb","openshift-image-registry/node-ca-4c2sk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal","openshift-multus/network-metrics-daemon-jphdx","openshift-network-diagnostics/network-check-target-k4jvq","kube-system/global-pull-secret-syncer-dmdfr","kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj","openshift-cluster-node-tuning-operator/tuned-jdp7p","openshift-multus/multus-additional-cni-plugins-2gtkn"]
Jul 03 14:11:40.004154 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.003969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.004154 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.004044    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:40.005394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.005367    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.006806 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.006783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.008464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008207    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 03 14:11:40.008464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008222    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 03 14:11:40.008464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008251    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fqws6\""
Jul 03 14:11:40.008464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008279    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.008734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008468    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.008734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008487    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.009190 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.008976    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 03 14:11:40.009596 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.009435    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 03 14:11:40.010006 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.009974    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 03 14:11:40.010312 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.010297    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.010485 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.010471    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.010640 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.010625    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 03 14:11:40.010793 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.010778    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 03 14:11:40.011229 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.011212    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vn8dl\""
Jul 03 14:11:40.011594 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.011577    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 03 14:11:40.011844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.011830    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4xjvt\""
Jul 03 14:11:40.012040 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.012027    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 03 14:11:40.012427 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.012410    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.012879 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.012859    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:40.012962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.012946    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.013244 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.012940    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:40.014602 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.014568    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.014840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.014823    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.015509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.015320    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.015509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.015402    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.015916    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-49vwk\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.016118    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.016298    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.016369    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.017187    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vlvrd\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.018710    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.018915    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z5qpf\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.019474    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.019789    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.020500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.020167    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.022012 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.021990    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.022937 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.022919    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.023153 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023103    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.023318 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023170    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zw47q\""
Jul 03 14:11:40.023504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.023607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-netd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.023670 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023625    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-socket-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.023670 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023660    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-bin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.023784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023700    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-etc-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.023784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023741    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.023893 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023788    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-cni-binary-copy\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.023893 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-os-release\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.023893 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023846    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.023893 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023854    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-systemd-units\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.024099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:40.024099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.023970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.024099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024004    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.024099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024048    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.024099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024080    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-sys-fs\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.024408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024134    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-hostroot\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.024408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024174    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-ovn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.024408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024208    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk6s8\" (UniqueName: \"kubernetes.io/projected/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kube-api-access-mk6s8\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.024408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-cnibin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.024408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024392    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfhcc\" (UniqueName: \"kubernetes.io/projected/cd0ac697-39d1-4245-81b9-a022dad293d4-kube-api-access-qfhcc\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024488    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-config\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024520    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-multus\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024561    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024602    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024634    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-daemon-config\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.024673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024662    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-log-socket\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-registration-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024791    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4t4x\" (UniqueName: \"kubernetes.io/projected/6c3d773e-c73c-4580-ad51-2843b424cb6e-kube-api-access-z4t4x\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57x28\" (UniqueName: \"kubernetes.io/projected/008fdfa1-c6cd-48de-8cae-5351451d35aa-kube-api-access-57x28\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024864    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-kubelet\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.025026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.024905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-system-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.025360 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025309    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a77bb5-d85b-4df5-b4ca-9f294ead9954-serviceca\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.025360 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025352    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhbn8\" (UniqueName: \"kubernetes.io/projected/80a77bb5-d85b-4df5-b4ca-9f294ead9954-kube-api-access-bhbn8\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.025469 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025372    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-dbus\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.025988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025962    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.026064 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025990    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lzg6r\""
Jul 03 14:11:40.026332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026315    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:11:40.026391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026334    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.026638 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.025423    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.026763 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-node-log\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.026825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026731    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-device-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.026825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-konnectivity-ca\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.026930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-os-release\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.026930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-cnibin\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.026930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026888    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ngkqf\""
Jul 03 14:11:40.026930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37993e17-a929-418e-b27b-c2f1c418ae10-ovn-node-metrics-cert\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-socket-dir-parent\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.026996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-k8s-cni-cncf-io\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027019    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 03 14:11:40.027131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027045    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-netns\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027376 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-kubelet\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027376 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027187    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.027376 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027246    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.027376 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-netns\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a77bb5-d85b-4df5-b4ca-9f294ead9954-host\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.027591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-slash\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027469    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 03 14:11:40.027591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-systemd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-env-overrides\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027577    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8lgn\" (UniqueName: \"kubernetes.io/projected/37993e17-a929-418e-b27b-c2f1c418ae10-kube-api-access-c8lgn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027629    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-script-lib\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.027834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027699    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-conf-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-multus-certs\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.027834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027800    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-etc-kubernetes\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.028097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027874    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-kubelet-config\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.028097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-var-lib-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.028097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-bin\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.028097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.027988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-agent-certs\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.028097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.028009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.062381 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.062331    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:06:39 +0000 UTC" deadline="2027-12-08 05:24:10.36116762 +0000 UTC"
Jul 03 14:11:40.062381 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.062381    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12543h12m30.298790418s"
Jul 03 14:11:40.115602 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.115571    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129170    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57x28\" (UniqueName: \"kubernetes.io/projected/008fdfa1-c6cd-48de-8cae-5351451d35aa-kube-api-access-57x28\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129229    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-kubelet\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-system-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129284    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a77bb5-d85b-4df5-b4ca-9f294ead9954-serviceca\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhbn8\" (UniqueName: \"kubernetes.io/projected/80a77bb5-d85b-4df5-b4ca-9f294ead9954-kube-api-access-bhbn8\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.129334 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-dbus\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129359    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-node-log\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129412    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-device-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129442    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-konnectivity-ca\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-os-release\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-cnibin\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129554    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37993e17-a929-418e-b27b-c2f1c418ae10-ovn-node-metrics-cert\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129580    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-socket-dir-parent\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-k8s-cni-cncf-io\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-netns\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129649    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-kubelet\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/834ce2f5-8690-4197-8785-4bbb549e3ab5-iptables-alerter-script\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129701    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-netns\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.129731 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a77bb5-d85b-4df5-b4ca-9f294ead9954-host\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129749    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-lib-modules\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129766    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-host\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129781    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-slash\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129797    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-systemd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-env-overrides\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8lgn\" (UniqueName: \"kubernetes.io/projected/37993e17-a929-418e-b27b-c2f1c418ae10-kube-api-access-c8lgn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e595515-da23-4809-b415-5f5ba0ee0014-tmp-dir\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495rw\" (UniqueName: \"kubernetes.io/projected/8e595515-da23-4809-b415-5f5ba0ee0014-kube-api-access-495rw\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129882    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-conf\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-script-lib\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129915    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-conf-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129933    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-multus-certs\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129962    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-etc-kubernetes\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129978    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-kubelet-config\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.129993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-run\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130009    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-var-lib-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.130510 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130024    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-bin\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130039    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-agent-certs\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130056    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-sys\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130084    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-tmp\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbqg\" (UniqueName: \"kubernetes.io/projected/834ce2f5-8690-4197-8785-4bbb549e3ab5-kube-api-access-xhbqg\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130118    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130134    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-netd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-socket-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130184    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-bin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130202    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-etc-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-cni-binary-copy\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130297    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-modprobe-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gmzz\" (UniqueName: \"kubernetes.io/projected/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-kube-api-access-5gmzz\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130352    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/834ce2f5-8690-4197-8785-4bbb549e3ab5-host-slash\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130381    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-os-release\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.131411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-systemd-units\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e595515-da23-4809-b415-5f5ba0ee0014-hosts-file\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130498    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-var-lib-kubelet\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130578    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130595    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-sys-fs\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130612    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-hostroot\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130631    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-systemd\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-ovn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130679    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk6s8\" (UniqueName: \"kubernetes.io/projected/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kube-api-access-mk6s8\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-cnibin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfhcc\" (UniqueName: \"kubernetes.io/projected/cd0ac697-39d1-4245-81b9-a022dad293d4-kube-api-access-qfhcc\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-config\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130778    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-multus\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132213 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysconfig\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130838    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-daemon-config\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130872    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-kubernetes\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130901    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-tuned\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130922    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-log-socket\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130940    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-registration-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130958    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.130980    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4t4x\" (UniqueName: \"kubernetes.io/projected/6c3d773e-c73c-4580-ad51-2843b424cb6e-kube-api-access-z4t4x\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.131406    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-kubelet\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.131468    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-system-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.131857    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a77bb5-d85b-4df5-b4ca-9f294ead9954-serviceca\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132146    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-dbus\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132294    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132342    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-node-log\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132351    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.132846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132398    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-device-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132399    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-netd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132487    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-socket-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132531    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-bin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132575    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-etc-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132633    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-cni-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.132960    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-konnectivity-ca\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133046    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-os-release\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133098    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-cnibin\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133138    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-cni-binary-copy\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133237    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-os-release\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133535    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-socket-dir-parent\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133576    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-k8s-cni-cncf-io\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133609    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-netns\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133645    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-kubelet\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.133676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133668    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-systemd-units\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.134524 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.133732    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:40.134524 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.133860    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:40.633838086 +0000 UTC m=+3.047734537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:40.134524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.133977    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/008fdfa1-c6cd-48de-8cae-5351451d35aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.134524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134031    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-run-netns\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.134524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134073    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a77bb5-d85b-4df5-b4ca-9f294ead9954-host\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.134762 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134645    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.134762 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134711    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-registration-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.134762 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134751    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.134853 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134799    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-sys-fs\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.134853 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134839    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-hostroot\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.134916 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.134875    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-ovn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.135150 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135117    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.135239 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135152    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-cnibin\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.135239 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135221    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07fdca58-6749-4863-8fcd-e302905155b2-kubelet-config\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.135373 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135290    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-conf-dir\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.135373 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135327    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-run-multus-certs\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.135373 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135360    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-etc-kubernetes\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.135581 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135541    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-var-lib-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.135652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135625    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-cni-bin\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.135743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135712    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 03 14:11:40.135782 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.135749    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.136366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.136343    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/008fdfa1-c6cd-48de-8cae-5351451d35aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.136934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.136770    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd0ac697-39d1-4245-81b9-a022dad293d4-host-var-lib-cni-multus\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.136934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.136914    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-config\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.137359 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.136841    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-openvswitch\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.137522 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.137494    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-log-socket\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.137613 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.136937    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.137613 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.137597    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd0ac697-39d1-4245-81b9-a022dad293d4-multus-daemon-config\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.137760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.137741    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-host-slash\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.137911 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.137830    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:40.137983 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.137971    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:11:40.63795137 +0000 UTC m=+3.051847815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:40.138051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.138006    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-ovnkube-script-lib\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.138140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.138065    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37993e17-a929-418e-b27b-c2f1c418ae10-run-systemd\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.138200 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.138159    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37993e17-a929-418e-b27b-c2f1c418ae10-env-overrides\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.141145 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.141122    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1078bd6c-b3c3-46e9-95e6-8e44a0561a95-agent-certs\") pod \"konnectivity-agent-clz2s\" (UID: \"1078bd6c-b3c3-46e9-95e6-8e44a0561a95\") " pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.141419 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.141402    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:40.141505 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.141429    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:40.141505 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.141446    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:40.141922 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.141505    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:40.641486806 +0000 UTC m=+3.055383251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:40.144587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.144562    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37993e17-a929-418e-b27b-c2f1c418ae10-ovn-node-metrics-cert\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.144693 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.144659    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhbn8\" (UniqueName: \"kubernetes.io/projected/80a77bb5-d85b-4df5-b4ca-9f294ead9954-kube-api-access-bhbn8\") pod \"node-ca-4c2sk\" (UID: \"80a77bb5-d85b-4df5-b4ca-9f294ead9954\") " pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.149322 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.149300    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:40.150531 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.150506    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfhcc\" (UniqueName: \"kubernetes.io/projected/cd0ac697-39d1-4245-81b9-a022dad293d4-kube-api-access-qfhcc\") pod \"multus-gnpsb\" (UID: \"cd0ac697-39d1-4245-81b9-a022dad293d4\") " pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.150664 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.150625    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57x28\" (UniqueName: \"kubernetes.io/projected/008fdfa1-c6cd-48de-8cae-5351451d35aa-kube-api-access-57x28\") pod \"multus-additional-cni-plugins-2gtkn\" (UID: \"008fdfa1-c6cd-48de-8cae-5351451d35aa\") " pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.150740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.150668    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4t4x\" (UniqueName: \"kubernetes.io/projected/6c3d773e-c73c-4580-ad51-2843b424cb6e-kube-api-access-z4t4x\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.150740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.150668    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8lgn\" (UniqueName: \"kubernetes.io/projected/37993e17-a929-418e-b27b-c2f1c418ae10-kube-api-access-c8lgn\") pod \"ovnkube-node-7rddn\" (UID: \"37993e17-a929-418e-b27b-c2f1c418ae10\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.154645 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.154616    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk6s8\" (UniqueName: \"kubernetes.io/projected/9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb-kube-api-access-mk6s8\") pod \"aws-ebs-csi-driver-node-hr6xj\" (UID: \"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.190878 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.190786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" event={"ID":"0e1adb89e96ffa1aea4eb2057cc3c519","Type":"ContainerStarted","Data":"25e9e516e8f5e10df56933c25f38bbcb9dc85f49d38a999967d550cf401240da"}
Jul 03 14:11:40.192112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.192077    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"954d8049aaecb57133d945db60ce4b63","Type":"ContainerStarted","Data":"cc163467ec2ec58d7d8c34c01e2f48d01695c47d804b4843c690957def741204"}
Jul 03 14:11:40.231913 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.231878    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-sys\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.231913 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.231911    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-tmp\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.231935    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbqg\" (UniqueName: \"kubernetes.io/projected/834ce2f5-8690-4197-8785-4bbb549e3ab5-kube-api-access-xhbqg\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.232128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232021    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-sys\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232076    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-modprobe-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232120    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gmzz\" (UniqueName: \"kubernetes.io/projected/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-kube-api-access-5gmzz\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/834ce2f5-8690-4197-8785-4bbb549e3ab5-host-slash\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232211    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e595515-da23-4809-b415-5f5ba0ee0014-hosts-file\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232237    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-var-lib-kubelet\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232262    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-modprobe-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232268    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-systemd\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232316    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-systemd\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232316    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e595515-da23-4809-b415-5f5ba0ee0014-hosts-file\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232316    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysconfig\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232264    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/834ce2f5-8690-4197-8785-4bbb549e3ab5-host-slash\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232351    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysconfig\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232380 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232375    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-var-lib-kubelet\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-kubernetes\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232424    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-tuned\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232434    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-kubernetes\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232461    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232492    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/834ce2f5-8690-4197-8785-4bbb549e3ab5-iptables-alerter-script\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232512    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-lib-modules\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232536    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-host\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232562    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e595515-da23-4809-b415-5f5ba0ee0014-tmp-dir\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232587    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-495rw\" (UniqueName: \"kubernetes.io/projected/8e595515-da23-4809-b415-5f5ba0ee0014-kube-api-access-495rw\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-conf\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232640    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-run\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.232788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232717    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-run\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232795    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-d\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232837    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-lib-modules\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.232880    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-host\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.233052    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e595515-da23-4809-b415-5f5ba0ee0014-tmp-dir\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.233075    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/834ce2f5-8690-4197-8785-4bbb549e3ab5-iptables-alerter-script\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.233607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.233055    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-sysctl-conf\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.234577 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.234558    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-tmp\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.235019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.235001    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-etc-tuned\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.239320 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.239300    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:11:40.244843 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.244755    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbqg\" (UniqueName: \"kubernetes.io/projected/834ce2f5-8690-4197-8785-4bbb549e3ab5-kube-api-access-xhbqg\") pod \"iptables-alerter-q585w\" (UID: \"834ce2f5-8690-4197-8785-4bbb549e3ab5\") " pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.244933 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.244847    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-495rw\" (UniqueName: \"kubernetes.io/projected/8e595515-da23-4809-b415-5f5ba0ee0014-kube-api-access-495rw\") pod \"node-resolver-778zb\" (UID: \"8e595515-da23-4809-b415-5f5ba0ee0014\") " pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.244933 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.244880    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gmzz\" (UniqueName: \"kubernetes.io/projected/4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e-kube-api-access-5gmzz\") pod \"tuned-jdp7p\" (UID: \"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e\") " pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.327291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.327250    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2gtkn"
Jul 03 14:11:40.336822 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.336794    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:11:40.338930 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.338888    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008fdfa1_c6cd_48de_8cae_5351451d35aa.slice/crio-92f0c431ceba47497831a9cf988d11178ad27a9d680f6ae208a7185c3a74bcef WatchSource:0}: Error finding container 92f0c431ceba47497831a9cf988d11178ad27a9d680f6ae208a7185c3a74bcef: Status 404 returned error can't find the container with id 92f0c431ceba47497831a9cf988d11178ad27a9d680f6ae208a7185c3a74bcef
Jul 03 14:11:40.346371 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.346344    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:11:40.346885 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.346858    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37993e17_a929_418e_b27b_c2f1c418ae10.slice/crio-d2715106428e628ca82857dc03ca640a96f258a96d5cc66726874a57fd763552 WatchSource:0}: Error finding container d2715106428e628ca82857dc03ca640a96f258a96d5cc66726874a57fd763552: Status 404 returned error can't find the container with id d2715106428e628ca82857dc03ca640a96f258a96d5cc66726874a57fd763552
Jul 03 14:11:40.355331 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.355307    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj"
Jul 03 14:11:40.362043 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.361892    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gnpsb"
Jul 03 14:11:40.362588 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.362561    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1078bd6c_b3c3_46e9_95e6_8e44a0561a95.slice/crio-e83429f4eb6dfe03a4ca14b8c1216e5cc19cc658f70980611cc81f4e2051baec WatchSource:0}: Error finding container e83429f4eb6dfe03a4ca14b8c1216e5cc19cc658f70980611cc81f4e2051baec: Status 404 returned error can't find the container with id e83429f4eb6dfe03a4ca14b8c1216e5cc19cc658f70980611cc81f4e2051baec
Jul 03 14:11:40.369553 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.369021    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9d82ef_3e45_4ef5_9ae6_7395ea1816bb.slice/crio-1b922623b8520cfb47f64c3e77129bf7362a04632e9070547423f20a2eae8dd8 WatchSource:0}: Error finding container 1b922623b8520cfb47f64c3e77129bf7362a04632e9070547423f20a2eae8dd8: Status 404 returned error can't find the container with id 1b922623b8520cfb47f64c3e77129bf7362a04632e9070547423f20a2eae8dd8
Jul 03 14:11:40.370015 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.369834    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4c2sk"
Jul 03 14:11:40.373442 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.373303    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0ac697_39d1_4245_81b9_a022dad293d4.slice/crio-99a7ea1d6a6668bcbaecb38417a6be549b601e661fcf1fb475064e19e0aa666e WatchSource:0}: Error finding container 99a7ea1d6a6668bcbaecb38417a6be549b601e661fcf1fb475064e19e0aa666e: Status 404 returned error can't find the container with id 99a7ea1d6a6668bcbaecb38417a6be549b601e661fcf1fb475064e19e0aa666e
Jul 03 14:11:40.380062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.380041    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-778zb"
Jul 03 14:11:40.380922 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.380897    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a77bb5_d85b_4df5_b4ca_9f294ead9954.slice/crio-12620a972ef24a04b0d43801d1bdee62d880612044a2dfce8bd8454e0c0af76f WatchSource:0}: Error finding container 12620a972ef24a04b0d43801d1bdee62d880612044a2dfce8bd8454e0c0af76f: Status 404 returned error can't find the container with id 12620a972ef24a04b0d43801d1bdee62d880612044a2dfce8bd8454e0c0af76f
Jul 03 14:11:40.388593 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.388424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jdp7p"
Jul 03 14:11:40.389455 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.389429    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e595515_da23_4809_b415_5f5ba0ee0014.slice/crio-9de44c1941522c51789dbdd2da85de41c6536f8980ed6944bacd71f479bdbed8 WatchSource:0}: Error finding container 9de44c1941522c51789dbdd2da85de41c6536f8980ed6944bacd71f479bdbed8: Status 404 returned error can't find the container with id 9de44c1941522c51789dbdd2da85de41c6536f8980ed6944bacd71f479bdbed8
Jul 03 14:11:40.395962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.395942    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q585w"
Jul 03 14:11:40.400000 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.399968    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2a1f68_fb7c_4e79_b1b3_7d4d88df1b0e.slice/crio-665a2fb039d7b84ddf31551ed092e7d2947d855dcb2def116d901d3d2b4e0d11 WatchSource:0}: Error finding container 665a2fb039d7b84ddf31551ed092e7d2947d855dcb2def116d901d3d2b4e0d11: Status 404 returned error can't find the container with id 665a2fb039d7b84ddf31551ed092e7d2947d855dcb2def116d901d3d2b4e0d11
Jul 03 14:11:40.407251 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:11:40.407225    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834ce2f5_8690_4197_8785_4bbb549e3ab5.slice/crio-438c6957ea5498ff402382dcd0f647f1dcd651b2f44301bc3800158d6ed40f24 WatchSource:0}: Error finding container 438c6957ea5498ff402382dcd0f647f1dcd651b2f44301bc3800158d6ed40f24: Status 404 returned error can't find the container with id 438c6957ea5498ff402382dcd0f647f1dcd651b2f44301bc3800158d6ed40f24
Jul 03 14:11:40.636263 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.636231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:40.636442 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.636370    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:40.636442 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.636436    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:41.636421319 +0000 UTC m=+4.050317748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:40.736961 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.736929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:40.737117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:40.736989    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:40.737117 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737099    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:40.737246 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737121    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:40.737246 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737132    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:40.737246 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737137    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:40.737246 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737187    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:41.737170715 +0000 UTC m=+4.151067144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:40.737246 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:40.737218    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:11:41.73721127 +0000 UTC m=+4.151107700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:41.063570 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.063418    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:06:39 +0000 UTC" deadline="2027-12-13 07:14:54.265819705 +0000 UTC"
Jul 03 14:11:41.063570 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.063487    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12665h3m13.202337464s"
Jul 03 14:11:41.184833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.184803    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:41.185038 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.184933    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:41.195648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.195607    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-778zb" event={"ID":"8e595515-da23-4809-b415-5f5ba0ee0014","Type":"ContainerStarted","Data":"9de44c1941522c51789dbdd2da85de41c6536f8980ed6944bacd71f479bdbed8"}
Jul 03 14:11:41.198073 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.198030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj" event={"ID":"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb","Type":"ContainerStarted","Data":"1b922623b8520cfb47f64c3e77129bf7362a04632e9070547423f20a2eae8dd8"}
Jul 03 14:11:41.199654 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.199626    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"d2715106428e628ca82857dc03ca640a96f258a96d5cc66726874a57fd763552"}
Jul 03 14:11:41.200977 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.200948    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q585w" event={"ID":"834ce2f5-8690-4197-8785-4bbb549e3ab5","Type":"ContainerStarted","Data":"438c6957ea5498ff402382dcd0f647f1dcd651b2f44301bc3800158d6ed40f24"}
Jul 03 14:11:41.202400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.202362    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jdp7p" event={"ID":"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e","Type":"ContainerStarted","Data":"665a2fb039d7b84ddf31551ed092e7d2947d855dcb2def116d901d3d2b4e0d11"}
Jul 03 14:11:41.203697 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.203649    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4c2sk" event={"ID":"80a77bb5-d85b-4df5-b4ca-9f294ead9954","Type":"ContainerStarted","Data":"12620a972ef24a04b0d43801d1bdee62d880612044a2dfce8bd8454e0c0af76f"}
Jul 03 14:11:41.205106 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.205050    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gnpsb" event={"ID":"cd0ac697-39d1-4245-81b9-a022dad293d4","Type":"ContainerStarted","Data":"99a7ea1d6a6668bcbaecb38417a6be549b601e661fcf1fb475064e19e0aa666e"}
Jul 03 14:11:41.206522 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.206474    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-clz2s" event={"ID":"1078bd6c-b3c3-46e9-95e6-8e44a0561a95","Type":"ContainerStarted","Data":"e83429f4eb6dfe03a4ca14b8c1216e5cc19cc658f70980611cc81f4e2051baec"}
Jul 03 14:11:41.207758 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.207721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerStarted","Data":"92f0c431ceba47497831a9cf988d11178ad27a9d680f6ae208a7185c3a74bcef"}
Jul 03 14:11:41.644193 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.643527    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:41.644193 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.643713    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:41.644193 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.643777    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:43.643758718 +0000 UTC m=+6.057655166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.744668    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:41.744730    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.744856    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.744922    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:11:43.7449024 +0000 UTC m=+6.158798834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.745376    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.745399    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.745413    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:41.745511 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:41.745461    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:43.745445296 +0000 UTC m=+6.159341728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:42.186426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:42.186392    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:42.186931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:42.186425    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:42.186931 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:42.186539    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:42.186931 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:42.186700    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:43.185760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.184995    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:43.185760 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.185400    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:43.232051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.232010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" event={"ID":"0e1adb89e96ffa1aea4eb2057cc3c519","Type":"ContainerStarted","Data":"9ec75f07d97dd6ff22f8d650de70e5a504c07fbbb64b5b1367afea6ab968f131"}
Jul 03 14:11:43.274516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.274356    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" podStartSLOduration=4.274334881 podStartE2EDuration="4.274334881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:11:43.272501411 +0000 UTC m=+5.686397867" watchObservedRunningTime="2026-07-03 14:11:43.274334881 +0000 UTC m=+5.688231335"
Jul 03 14:11:43.662547 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.662504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:43.662752 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.662675    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:43.662752 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.662742    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:47.662721976 +0000 UTC m=+10.076618412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:43.763912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.763867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:43.764109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:43.763948    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:43.764169 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764114    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:43.764209 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764183    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:11:47.764163253 +0000 UTC m=+10.178059697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:43.764701 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764680    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:43.764798 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764707    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:43.764798 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764722    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:43.764798 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:43.764775    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:47.76475817 +0000 UTC m=+10.178654618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:44.187097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:44.186385    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:44.187097 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:44.186556    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:44.187097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:44.186984    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:44.187097 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:44.187060    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:44.294225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:44.293086    2561 generic.go:356] "Generic (PLEG): container finished" podID="954d8049aaecb57133d945db60ce4b63" containerID="76f2a8e49f7c2635a52e9afa0e21779c08e567898be40c0bc71a9b40df0db160" exitCode=0
Jul 03 14:11:44.294225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:44.294169    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"954d8049aaecb57133d945db60ce4b63","Type":"ContainerDied","Data":"76f2a8e49f7c2635a52e9afa0e21779c08e567898be40c0bc71a9b40df0db160"}
Jul 03 14:11:45.184781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:45.184689    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:45.184962 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:45.184835    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:46.185373 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:46.185341    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:46.185862 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:46.185513    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:46.186055 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:46.186033    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:46.186143 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:46.186124    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:47.185314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:47.185276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:47.185519 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.185408    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:47.703005 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:47.702967    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:47.703162 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.703136    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:47.703239 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.703212    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:55.703188743 +0000 UTC m=+18.117085196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:47.804158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:47.804269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804433    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804451    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804463    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804521    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:11:55.80450156 +0000 UTC m=+18.218398008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804938    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:47.805021 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:47.804995    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:11:55.804976885 +0000 UTC m=+18.218873318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:48.185591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:48.185543    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:48.186029 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:48.185668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:48.186029 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:48.185743    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:48.186029 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:48.185762    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:49.185294 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:49.185229    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:49.185465 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:49.185343    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:50.185309 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:50.185275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:50.185928 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:50.185276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:50.185928 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:50.185421    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:50.185928 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:50.185477    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:51.185352 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:51.185325    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:51.185787 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:51.185434    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:52.185195 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:52.185159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:52.185297 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:52.185161    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:52.185372 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:52.185274    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:52.185627 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:52.185396    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:53.185201 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:53.185151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:53.185379 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:53.185300    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:54.185278 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:54.185233    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:54.185278 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:54.185276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:54.185727 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:54.185349    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:54.185727 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:54.185486    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:55.185379 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:55.185339    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:55.185599 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.185460    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:55.768655 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:55.768617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:55.768803 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.768753    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:55.768846 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.768826    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:11.768802172 +0000 UTC m=+34.182698614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:11:55.868939 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:55.868906    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:55.868951    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869046    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869063    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869080    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869090    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:55.869103 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869097    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:12:11.869082312 +0000 UTC m=+34.282978740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:11:55.869335 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:55.869130    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:11.869118557 +0000 UTC m=+34.283015002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:11:56.185318 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:56.185290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:56.185436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:56.185336    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:56.185436 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:56.185419    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:56.185553 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:56.185537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:57.184667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:57.184639    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:57.185026 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:57.184748    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:11:58.185859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:58.185807    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:11:58.186312 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:58.185934    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:11:58.186312 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:58.185999    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:11:58.186312 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:58.186117    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:11:59.184952 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:11:59.184913    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:11:59.185108 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:11:59.185051    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:00.184546 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:00.184512    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:00.185116 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:00.184516    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:00.185116 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:00.184651    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:00.185116 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:00.184719    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:01.185282 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.184969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:01.186053 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:01.185360    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:01.329083 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.329042    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"954d8049aaecb57133d945db60ce4b63","Type":"ContainerStarted","Data":"06b7ca6b25d529d5901346ef9a5ec6e2947ddb86bcb6390f05fdffcd9476e926"}
Jul 03 14:12:01.330851 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.330820    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jdp7p" event={"ID":"4b2a1f68-fb7c-4e79-b1b3-7d4d88df1b0e","Type":"ContainerStarted","Data":"8ef9c836525036c283c018b1dbb1d43eba8ca0accb3e3866354863842e3f2fc3"}
Jul 03 14:12:01.332913 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.332700    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4c2sk" event={"ID":"80a77bb5-d85b-4df5-b4ca-9f294ead9954","Type":"ContainerStarted","Data":"ce7d6c9b10a83647caa0fa271c835656dd45f07981c0f60e6cfc95189f246e60"}
Jul 03 14:12:01.334306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.334284    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gnpsb" event={"ID":"cd0ac697-39d1-4245-81b9-a022dad293d4","Type":"ContainerStarted","Data":"75424d9ea172fa6849531d3b0cc709413883dfa64305690a011f0ce6499d688c"}
Jul 03 14:12:01.336230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.336208    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-clz2s" event={"ID":"1078bd6c-b3c3-46e9-95e6-8e44a0561a95","Type":"ContainerStarted","Data":"9b2a19b5e5c28a5eacf13323efd1e5cc090501b3fe3fe48019e71e83a0c9f19a"}
Jul 03 14:12:01.337855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.337833    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="87d6b65b46891d8fa37ea7ace669a1d92e7d5b02a693587ca371299588e589ea" exitCode=0
Jul 03 14:12:01.337960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.337912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"87d6b65b46891d8fa37ea7ace669a1d92e7d5b02a693587ca371299588e589ea"}
Jul 03 14:12:01.339983 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.339959    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-778zb" event={"ID":"8e595515-da23-4809-b415-5f5ba0ee0014","Type":"ContainerStarted","Data":"1d23d6734be31b35ec7f0963f12bf8396c2916ac5af4f2c7b274fba9335bff45"}
Jul 03 14:12:01.341834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.341812    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj" event={"ID":"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb","Type":"ContainerStarted","Data":"f66e59e6a2470efe3bafafcc59c535c09712154e4c346ade14b536a7c01a0dab"}
Jul 03 14:12:01.345350 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345326    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"8ee71f2a9e81c38e31b1245ac27c792a29f857b156e5fd2ad9b0e7c47ac4f9f4"}
Jul 03 14:12:01.345448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345398    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"e5961740cbdcb285cc0efc163b8e454f966b65e4bd7725831020cd803031a9a2"}
Jul 03 14:12:01.345491 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"d0a9ae9dea4cec33d31770476601fdfda34cf2448e1f8120bd91398d18254b56"}
Jul 03 14:12:01.345532 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345506    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"24f63f12b6cab22e0e215815b7f4230cc0461352758963ca4dc25918e8ffb071"}
Jul 03 14:12:01.345581 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345554    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"a4bb7d4bbb494ac02f56300c604521151f32446517fa61e18219105c7c218e9d"}
Jul 03 14:12:01.345631 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.345612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"d32a681d3b876bc12b8b7605030288743cb6997cf500c3c592441f5e407e217c"}
Jul 03 14:12:01.348110 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.348068    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" podStartSLOduration=22.34805163 podStartE2EDuration="22.34805163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.347597657 +0000 UTC m=+23.761494120" watchObservedRunningTime="2026-07-03 14:12:01.34805163 +0000 UTC m=+23.761948081"
Jul 03 14:12:01.364082 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.364036    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4c2sk" podStartSLOduration=2.229238101 podStartE2EDuration="22.364018867s" totalImagesPullingTime="20.134780766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.363400406 +0000 UTC m=+23.777296869" watchObservedRunningTime="2026-07-03 14:12:01.364018867 +0000 UTC m=+23.777915318"
Jul 03 14:12:01.379363 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.379323    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-778zb" podStartSLOduration=2.295744846 podStartE2EDuration="22.379308732s" totalImagesPullingTime="20.083563886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.37856573 +0000 UTC m=+23.792462181" watchObservedRunningTime="2026-07-03 14:12:01.379308732 +0000 UTC m=+23.793205183"
Jul 03 14:12:01.395975 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.395938    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jdp7p" podStartSLOduration=2.279667787 podStartE2EDuration="22.395923732s" totalImagesPullingTime="20.116255945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.395671727 +0000 UTC m=+23.809568194" watchObservedRunningTime="2026-07-03 14:12:01.395923732 +0000 UTC m=+23.809820183"
Jul 03 14:12:01.432271 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.432225    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-clz2s" podStartSLOduration=7.904425302 podStartE2EDuration="23.43220808s" totalImagesPullingTime="15.527782778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.431916406 +0000 UTC m=+23.845812855" watchObservedRunningTime="2026-07-03 14:12:01.43220808 +0000 UTC m=+23.846104531"
Jul 03 14:12:01.450357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:01.450307    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gnpsb" podStartSLOduration=3.237279674 podStartE2EDuration="23.450292031s" totalImagesPullingTime="20.213012357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:01.450064804 +0000 UTC m=+23.863961256" watchObservedRunningTime="2026-07-03 14:12:01.450292031 +0000 UTC m=+23.864188481"
Jul 03 14:12:02.041157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.041130    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 03 14:12:02.088206 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.088039    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T14:12:02.04115109Z","UUID":"1a8947c3-899a-4b06-a211-932a554f5853","Handler":null,"Name":"","Endpoint":""}
Jul 03 14:12:02.089883 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.089860    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 03 14:12:02.089984 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.089901    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 03 14:12:02.184527 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.184498    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:02.184692 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.184537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:02.184692 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:02.184619    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:02.184765 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:02.184707    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:02.348664 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.348633    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj" event={"ID":"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb","Type":"ContainerStarted","Data":"c070137f558b2b60922958965553810cf1a53ed52bbeae77dd6566e870c069fc"}
Jul 03 14:12:02.350007 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.349984    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q585w" event={"ID":"834ce2f5-8690-4197-8785-4bbb549e3ab5","Type":"ContainerStarted","Data":"0f66d5517e976d3709af765e0e2fd6f193470a7623f95299fe3d9573a63af3cb"}
Jul 03 14:12:02.369735 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:02.369696    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q585w" podStartSLOduration=3.303805635 podStartE2EDuration="23.369684517s" totalImagesPullingTime="20.065878882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:02.369587621 +0000 UTC m=+24.783484072" watchObservedRunningTime="2026-07-03 14:12:02.369684517 +0000 UTC m=+24.783580967"
Jul 03 14:12:03.184540 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:03.184510    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:03.184773 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:03.184611    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:04.184834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.184669    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:04.185173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.184669    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:04.185173 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:04.184923    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:04.185173 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:04.184972    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:04.355435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.355399    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj" event={"ID":"9c9d82ef-3e45-4ef5-9ae6-7395ea1816bb","Type":"ContainerStarted","Data":"79810f09068420355374a4cc4045611a6fa6ad3e059f9cdb12094df2841bc9e4"}
Jul 03 14:12:04.358632 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.358611    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"c6752d2a18b7894faac432eb47ce80cfdcc79eb3a11de001c846e38b5568386a"}
Jul 03 14:12:04.377829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.377786    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hr6xj" podStartSLOduration=3.677075349 podStartE2EDuration="26.377771813s" totalImagesPullingTime="22.700696464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:04.377631791 +0000 UTC m=+26.791528234" watchObservedRunningTime="2026-07-03 14:12:04.377771813 +0000 UTC m=+26.791668263"
Jul 03 14:12:04.513395 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.513307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:12:04.513864 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:04.513842    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:12:05.185448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:05.185421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:05.185841 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:05.185537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:05.361170 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:05.361105    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:12:05.361588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:05.361563    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-clz2s"
Jul 03 14:12:06.184576 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:06.184544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:06.184717 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:06.184684    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:06.184717 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:06.184714    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:06.184813 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:06.184771    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:06.367940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:06.367699    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" event={"ID":"37993e17-a929-418e-b27b-c2f1c418ae10","Type":"ContainerStarted","Data":"4e21ad60103a40f23ad4caadb9779c2d557dad504948098774213e0433ba301b"}
Jul 03 14:12:06.397370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:06.397324    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn" podStartSLOduration=8.131470654 podStartE2EDuration="28.397308987s" totalImagesPullingTime="20.265838333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:06.396735063 +0000 UTC m=+28.810631514" watchObservedRunningTime="2026-07-03 14:12:06.397308987 +0000 UTC m=+28.811205436"
Jul 03 14:12:07.184456 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.184428    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:07.184606 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:07.184549    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:07.371069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.370524    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:07.371069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.370612    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:07.371069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.370653    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:07.390417 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.390375    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:07.390586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.390575    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:07.926702 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.926669    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dmdfr"]
Jul 03 14:12:07.926900 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.926829    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:07.926963 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:07.926929    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:07.930141 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.930109    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k4jvq"]
Jul 03 14:12:07.930302 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.930245    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:07.930376 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:07.930343    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:07.930721 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.930694    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jphdx"]
Jul 03 14:12:07.930812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:07.930788    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:07.930907 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:07.930869    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:09.185105 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:09.185088    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:09.185359 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:09.185203    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:09.379239 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:09.379191    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerStarted","Data":"c35648b2ab46b8d80bc0e9c1b297578965147c49266ba11b9b2471be23b4c8ca"}
Jul 03 14:12:10.188545 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:10.188360    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:10.188885 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:10.188412    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:10.188885 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:10.188629    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:10.188885 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:10.188685    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:10.383485 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:10.383331    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="c35648b2ab46b8d80bc0e9c1b297578965147c49266ba11b9b2471be23b4c8ca" exitCode=0
Jul 03 14:12:10.383485 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:10.383380    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"c35648b2ab46b8d80bc0e9c1b297578965147c49266ba11b9b2471be23b4c8ca"}
Jul 03 14:12:11.184734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.184709    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:11.184857 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.184833    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:11.386930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.386901    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="d53b927c99d1829aabb35246b650513271ddca60872ef4e5451ee837a65f115e" exitCode=0
Jul 03 14:12:11.387445 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.386935    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"d53b927c99d1829aabb35246b650513271ddca60872ef4e5451ee837a65f115e"}
Jul 03 14:12:11.786492 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.786461    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:11.786643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.786580    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:12:11.786643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.786640    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret podName:07fdca58-6749-4863-8fcd-e302905155b2 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:43.786623022 +0000 UTC m=+66.200519451 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret") pod "global-pull-secret-syncer-dmdfr" (UID: "07fdca58-6749-4863-8fcd-e302905155b2") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:12:11.887252 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.887208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:11.887397 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:11.887301    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:11.887397 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887377    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:12:11.887496 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887436    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:12:43.887419647 +0000 UTC m=+66.301316078 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:12:11.887496 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887447    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:12:11.887496 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887464    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:12:11.887496 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887476    2561 projected.go:196] Error preparing data for projected volume kube-api-access-9v9px for pod openshift-network-diagnostics/network-check-target-k4jvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:12:11.887646 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:11.887524    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px podName:1b329db1-90a9-4bb9-941d-a23411f47c16 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:43.887509799 +0000 UTC m=+66.301406236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v9px" (UniqueName: "kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px") pod "network-check-target-k4jvq" (UID: "1b329db1-90a9-4bb9-941d-a23411f47c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:12:12.184837 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:12.184807    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:12.184984 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:12.184846    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:12.184984 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:12.184933    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:12.185084 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:12.185043    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:12.391297 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:12.391262    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="4576c8afce3dec8841c05a7430cc9968fd3157185e7f1b5cbe7a282ef832149e" exitCode=0
Jul 03 14:12:12.391745 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:12.391312    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"4576c8afce3dec8841c05a7430cc9968fd3157185e7f1b5cbe7a282ef832149e"}
Jul 03 14:12:13.185357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:13.185322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:13.185520 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:13.185440    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmdfr" podUID="07fdca58-6749-4863-8fcd-e302905155b2"
Jul 03 14:12:14.185889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.185445    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:14.185889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.185493    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:14.185889 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:14.185592    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jphdx" podUID="6c3d773e-c73c-4580-ad51-2843b424cb6e"
Jul 03 14:12:14.185889 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:14.185745    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k4jvq" podUID="1b329db1-90a9-4bb9-941d-a23411f47c16"
Jul 03 14:12:14.320939 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.320867    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeReady"
Jul 03 14:12:14.321067 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.320974    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 03 14:12:14.367517 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.367467    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hlpdf"]
Jul 03 14:12:14.369404 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.369386    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.372336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.372316    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 03 14:12:14.372336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.372330    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xht5n\""
Jul 03 14:12:14.372520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.372331    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 03 14:12:14.380632 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.380611    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rmgm9"]
Jul 03 14:12:14.382362 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.382342    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.383744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.383724    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hlpdf"]
Jul 03 14:12:14.385283 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.385258    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gqn8m\""
Jul 03 14:12:14.385377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.385338    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 03 14:12:14.385377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.385374    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 03 14:12:14.385565 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.385490    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 03 14:12:14.391675 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.391657    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rmgm9"]
Jul 03 14:12:14.504448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504415    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494eba7d-34c7-493e-8829-84bfaefb21d5-cert\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.504448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504447    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k77z5\" (UniqueName: \"kubernetes.io/projected/406123bb-f445-4919-b537-dd4c8049c717-kube-api-access-k77z5\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.504643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504470    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.504643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504577    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/406123bb-f445-4919-b537-dd4c8049c717-config-volume\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.504643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504601    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406123bb-f445-4919-b537-dd4c8049c717-tmp-dir\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.504643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.504624    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4m6j\" (UniqueName: \"kubernetes.io/projected/494eba7d-34c7-493e-8829-84bfaefb21d5-kube-api-access-x4m6j\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.605325 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605360    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/406123bb-f445-4919-b537-dd4c8049c717-config-volume\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605384    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406123bb-f445-4919-b537-dd4c8049c717-tmp-dir\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4m6j\" (UniqueName: \"kubernetes.io/projected/494eba7d-34c7-493e-8829-84bfaefb21d5-kube-api-access-x4m6j\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494eba7d-34c7-493e-8829-84bfaefb21d5-cert\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:14.605442    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:14.605466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k77z5\" (UniqueName: \"kubernetes.io/projected/406123bb-f445-4919-b537-dd4c8049c717-kube-api-access-k77z5\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.605747 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:14.605513    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:15.1054904 +0000 UTC m=+37.519386842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:14.605795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605783    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406123bb-f445-4919-b537-dd4c8049c717-tmp-dir\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.605998 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.605982    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/406123bb-f445-4919-b537-dd4c8049c717-config-volume\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.609400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.609384    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494eba7d-34c7-493e-8829-84bfaefb21d5-cert\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.616767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.616745    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k77z5\" (UniqueName: \"kubernetes.io/projected/406123bb-f445-4919-b537-dd4c8049c717-kube-api-access-k77z5\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:14.616879 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.616863    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4m6j\" (UniqueName: \"kubernetes.io/projected/494eba7d-34c7-493e-8829-84bfaefb21d5-kube-api-access-x4m6j\") pod \"ingress-canary-rmgm9\" (UID: \"494eba7d-34c7-493e-8829-84bfaefb21d5\") " pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.694744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.694717    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rmgm9"
Jul 03 14:12:14.860058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:14.859779    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rmgm9"]
Jul 03 14:12:14.865772 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:14.865741    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494eba7d_34c7_493e_8829_84bfaefb21d5.slice/crio-f3f82fc1e042b4906d1a7a36535179bfc014c51adaebf6691c800601c5380666 WatchSource:0}: Error finding container f3f82fc1e042b4906d1a7a36535179bfc014c51adaebf6691c800601c5380666: Status 404 returned error can't find the container with id f3f82fc1e042b4906d1a7a36535179bfc014c51adaebf6691c800601c5380666
Jul 03 14:12:15.108643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:15.108614    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:15.108827 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:15.108768    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:15.108889 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:15.108838    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:16.108815003 +0000 UTC m=+38.522711449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:15.185030 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:15.184965    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:15.187655 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:15.187633    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 14:12:15.398872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:15.398834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rmgm9" event={"ID":"494eba7d-34c7-493e-8829-84bfaefb21d5","Type":"ContainerStarted","Data":"f3f82fc1e042b4906d1a7a36535179bfc014c51adaebf6691c800601c5380666"}
Jul 03 14:12:16.115644 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.115615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:16.115818 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:16.115797    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:16.115888 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:16.115875    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:18.115855364 +0000 UTC m=+40.529751797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:16.189784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.189758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:16.190155 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.190127    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:16.193461 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.193440    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m5fkc\""
Jul 03 14:12:16.193593 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.193476    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 14:12:16.193652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.193624    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 14:12:16.193727 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.193709    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 14:12:16.193892 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:16.193872    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kg466\""
Jul 03 14:12:18.133936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:18.133897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:18.134402 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:18.134062    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:18.134402 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:18.134136    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:22.134113583 +0000 UTC m=+44.548010011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:19.412028 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:19.411993    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="68e36d7bdcd4bef3a96224c9b212d245b8af11faf097dde6ef13ea9b684a9799" exitCode=0
Jul 03 14:12:19.412417 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:19.412078    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"68e36d7bdcd4bef3a96224c9b212d245b8af11faf097dde6ef13ea9b684a9799"}
Jul 03 14:12:19.413565 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:19.413538    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rmgm9" event={"ID":"494eba7d-34c7-493e-8829-84bfaefb21d5","Type":"ContainerStarted","Data":"1ecc222e3698463e27cfdd8bef6adec347c6dcae9757546c363365ca093dd4de"}
Jul 03 14:12:19.453720 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:19.453678    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rmgm9" podStartSLOduration=1.662574571 podStartE2EDuration="5.453663375s" totalImagesPullingTime="3.791088804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:19.452704074 +0000 UTC m=+41.866600524" watchObservedRunningTime="2026-07-03 14:12:19.453663375 +0000 UTC m=+41.867559829"
Jul 03 14:12:20.419146 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:20.419114    2561 generic.go:356] "Generic (PLEG): container finished" podID="008fdfa1-c6cd-48de-8cae-5351451d35aa" containerID="16fc9eb9617e7ed418ca291a34b06845e7f8d435eb82c509e2df4bbbfbf90f9a" exitCode=0
Jul 03 14:12:20.419520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:20.419200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerDied","Data":"16fc9eb9617e7ed418ca291a34b06845e7f8d435eb82c509e2df4bbbfbf90f9a"}
Jul 03 14:12:21.425102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:21.425068    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" event={"ID":"008fdfa1-c6cd-48de-8cae-5351451d35aa","Type":"ContainerStarted","Data":"39fe3212fe7a32f23e91f527c3bf1fdd8995ade33a43ef8d4cebeed21bb46fb0"}
Jul 03 14:12:21.450426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:21.450384    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2gtkn" podStartSLOduration=7.84105377 podStartE2EDuration="43.450369876s" totalImagesPullingTime="35.609316106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:21.448564808 +0000 UTC m=+43.862461258" watchObservedRunningTime="2026-07-03 14:12:21.450369876 +0000 UTC m=+43.864266327"
Jul 03 14:12:22.158547 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.158514    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:22.158704 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:22.158633    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:22.158704 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:22.158683    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:30.158670493 +0000 UTC m=+52.572566921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:22.261408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.261378    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cj294"]
Jul 03 14:12:22.277994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.277970    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cj294"]
Jul 03 14:12:22.278112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.278056    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.280845 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.280823    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 03 14:12:22.280946 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.280857    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 03 14:12:22.282133 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.282118    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 03 14:12:22.282211 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.282144    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xqz29\""
Jul 03 14:12:22.283370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.283305    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:12:22.287414 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.287398    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 03 14:12:22.359557 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.359527    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"]
Jul 03 14:12:22.382473 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.382453    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"]
Jul 03 14:12:22.382558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.382533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"
Jul 03 14:12:22.385368 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.385351    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:12:22.385472 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.385353    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 03 14:12:22.385472 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.385407    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xtw4c\""
Jul 03 14:12:22.460498 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.460433    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-trusted-ca\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.460498 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.460473    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-config\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.460818 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.460496    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-serving-cert\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.460818 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.460547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghc5h\" (UniqueName: \"kubernetes.io/projected/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-kube-api-access-ghc5h\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.561787 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.561753    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlnw\" (UniqueName: \"kubernetes.io/projected/c57f3681-87b3-4c03-ac6b-689fc2bf174b-kube-api-access-mhlnw\") pod \"volume-data-source-validator-5d456fdbcd-mvg7r\" (UID: \"c57f3681-87b3-4c03-ac6b-689fc2bf174b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"
Jul 03 14:12:22.561917 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.561896    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-trusted-ca\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.561997 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.561960    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-config\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.561997 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.561984    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-serving-cert\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.562094 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.562049    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghc5h\" (UniqueName: \"kubernetes.io/projected/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-kube-api-access-ghc5h\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.562749 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.562727    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-config\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.563139 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.563123    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-trusted-ca\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.565656 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.565636    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-serving-cert\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.573485 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.573461    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghc5h\" (UniqueName: \"kubernetes.io/projected/58d23fb7-3fe9-4b16-8719-a155bf7fa66c-kube-api-access-ghc5h\") pod \"console-operator-68b757fb67-cj294\" (UID: \"58d23fb7-3fe9-4b16-8719-a155bf7fa66c\") " pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.588441 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.588424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:22.663110 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.663083    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlnw\" (UniqueName: \"kubernetes.io/projected/c57f3681-87b3-4c03-ac6b-689fc2bf174b-kube-api-access-mhlnw\") pod \"volume-data-source-validator-5d456fdbcd-mvg7r\" (UID: \"c57f3681-87b3-4c03-ac6b-689fc2bf174b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"
Jul 03 14:12:22.671850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.671774    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:12:22.680235 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.680208    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlnw\" (UniqueName: \"kubernetes.io/projected/c57f3681-87b3-4c03-ac6b-689fc2bf174b-kube-api-access-mhlnw\") pod \"volume-data-source-validator-5d456fdbcd-mvg7r\" (UID: \"c57f3681-87b3-4c03-ac6b-689fc2bf174b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"
Jul 03 14:12:22.691241 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.691217    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"
Jul 03 14:12:22.706741 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.706719    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:12:22.706859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.706828    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.709255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.709233    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 03 14:12:22.709255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.709244    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkp7q\""
Jul 03 14:12:22.709415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.709235    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 03 14:12:22.709415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.709285    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 03 14:12:22.714575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.714554    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 03 14:12:22.733047 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.732993    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cj294"]
Jul 03 14:12:22.737030 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:22.736999    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d23fb7_3fe9_4b16_8719_a155bf7fa66c.slice/crio-8a2703b88be4eaa5258795b3f9a678ff5535ee9b5971a4fc96ccbdfcb2b0eb23 WatchSource:0}: Error finding container 8a2703b88be4eaa5258795b3f9a678ff5535ee9b5971a4fc96ccbdfcb2b0eb23: Status 404 returned error can't find the container with id 8a2703b88be4eaa5258795b3f9a678ff5535ee9b5971a4fc96ccbdfcb2b0eb23
Jul 03 14:12:22.839307 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.839281    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r"]
Jul 03 14:12:22.842429 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:22.842393    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57f3681_87b3_4c03_ac6b_689fc2bf174b.slice/crio-874f387c36050eeb5ae754953618073b738d1c3fa9fccdb715ae743486fdbfef WatchSource:0}: Error finding container 874f387c36050eeb5ae754953618073b738d1c3fa9fccdb715ae743486fdbfef: Status 404 returned error can't find the container with id 874f387c36050eeb5ae754953618073b738d1c3fa9fccdb715ae743486fdbfef
Jul 03 14:12:22.865137 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865150    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865228    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7rsd\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865398    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.865531 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.865438    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966209 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7rsd\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966209 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966209    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966289    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966319    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966574 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.966372    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.966574 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:22.966379    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:22.966574 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:22.966398    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:22.966574 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:22.966500    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:23.466463796 +0000 UTC m=+45.880360237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:22.967034 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.967005    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.967289 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.967273    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.967336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.967284    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.968757 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.968739    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.968841 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.968790    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.977205 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.977166    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7rsd\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:22.977267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:22.977221    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:23.432474 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:23.432433    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r" event={"ID":"c57f3681-87b3-4c03-ac6b-689fc2bf174b","Type":"ContainerStarted","Data":"874f387c36050eeb5ae754953618073b738d1c3fa9fccdb715ae743486fdbfef"}
Jul 03 14:12:23.433728 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:23.433699    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cj294" event={"ID":"58d23fb7-3fe9-4b16-8719-a155bf7fa66c","Type":"ContainerStarted","Data":"8a2703b88be4eaa5258795b3f9a678ff5535ee9b5971a4fc96ccbdfcb2b0eb23"}
Jul 03 14:12:23.470278 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:23.470251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:23.470609 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:23.470422    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:23.470609 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:23.470438    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:23.470609 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:23.470489    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:24.470474208 +0000 UTC m=+46.884370636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:24.479654 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:24.479621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:24.480002 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:24.479744    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:24.480002 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:24.479755    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:24.480002 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:24.479807    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:26.47979049 +0000 UTC m=+48.893686923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:25.438708 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:25.438684    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r" event={"ID":"c57f3681-87b3-4c03-ac6b-689fc2bf174b","Type":"ContainerStarted","Data":"20f490eaae57c1974388c1caee40e47ae7b095c54af977bdff37dfaf200c55e0"}
Jul 03 14:12:25.472335 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:25.472286    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mvg7r" podStartSLOduration=1.193552742 podStartE2EDuration="3.472266223s" totalImagesPullingTime="2.278713481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:25.471966938 +0000 UTC m=+47.885863389" watchObservedRunningTime="2026-07-03 14:12:25.472266223 +0000 UTC m=+47.886162667"
Jul 03 14:12:26.442589 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:26.442561    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/0.log"
Jul 03 14:12:26.442949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:26.442603    2561 generic.go:356] "Generic (PLEG): container finished" podID="58d23fb7-3fe9-4b16-8719-a155bf7fa66c" containerID="9e8a6cdda410ab7c3cc9cfc035714d674f706fac981c38d716f2110ae12de4bf" exitCode=255
Jul 03 14:12:26.442949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:26.442695    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cj294" event={"ID":"58d23fb7-3fe9-4b16-8719-a155bf7fa66c","Type":"ContainerDied","Data":"9e8a6cdda410ab7c3cc9cfc035714d674f706fac981c38d716f2110ae12de4bf"}
Jul 03 14:12:26.443016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:26.443000    2561 scope.go:117] "RemoveContainer" containerID="9e8a6cdda410ab7c3cc9cfc035714d674f706fac981c38d716f2110ae12de4bf"
Jul 03 14:12:26.495920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:26.495898    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:26.496046 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:26.496029    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:26.496090 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:26.496048    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:26.496120 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:26.496095    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:30.4960812 +0000 UTC m=+52.909977629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:27.446708 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.446682    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:12:27.447090 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.447040    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/0.log"
Jul 03 14:12:27.447090 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.447075    2561 generic.go:356] "Generic (PLEG): container finished" podID="58d23fb7-3fe9-4b16-8719-a155bf7fa66c" containerID="e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9" exitCode=255
Jul 03 14:12:27.447164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.447112    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cj294" event={"ID":"58d23fb7-3fe9-4b16-8719-a155bf7fa66c","Type":"ContainerDied","Data":"e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9"}
Jul 03 14:12:27.447216 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.447165    2561 scope.go:117] "RemoveContainer" containerID="9e8a6cdda410ab7c3cc9cfc035714d674f706fac981c38d716f2110ae12de4bf"
Jul 03 14:12:27.447423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.447402    2561 scope.go:117] "RemoveContainer" containerID="e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9"
Jul 03 14:12:27.447612 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:27.447593    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cj294_openshift-console-operator(58d23fb7-3fe9-4b16-8719-a155bf7fa66c)\"" pod="openshift-console-operator/console-operator-68b757fb67-cj294" podUID="58d23fb7-3fe9-4b16-8719-a155bf7fa66c"
Jul 03 14:12:27.702400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.702319    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxv2"]
Jul 03 14:12:27.731249 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.731226    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxv2"]
Jul 03 14:12:27.731404 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.731322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:27.733560 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.733521    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 03 14:12:27.733560 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.733526    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pz6z4\""
Jul 03 14:12:27.733779 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.733763    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 03 14:12:27.733849 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.733770    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 03 14:12:27.733998 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.733984    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 03 14:12:27.906520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.906492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af900b5e-358a-481c-a359-9844bccc514c-crio-socket\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:27.906639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.906560    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9nmd\" (UniqueName: \"kubernetes.io/projected/af900b5e-358a-481c-a359-9844bccc514c-kube-api-access-f9nmd\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:27.906639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.906582    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af900b5e-358a-481c-a359-9844bccc514c-data-volume\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:27.906639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.906606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:27.906639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:27.906628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007543    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af900b5e-358a-481c-a359-9844bccc514c-crio-socket\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9nmd\" (UniqueName: \"kubernetes.io/projected/af900b5e-358a-481c-a359-9844bccc514c-kube-api-access-f9nmd\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af900b5e-358a-481c-a359-9844bccc514c-data-volume\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007686    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007918 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007787    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af900b5e-358a-481c-a359-9844bccc514c-crio-socket\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.007918 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.007800    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:12:28.007918 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.007823    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:28.007918 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.007863    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:28.507849684 +0000 UTC m=+50.921746132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : configmap "kube-rbac-proxy" not found
Jul 03 14:12:28.007918 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.007898    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:28.507878623 +0000 UTC m=+50.921775053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:28.008074 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.007951    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af900b5e-358a-481c-a359-9844bccc514c-data-volume\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.016750 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.016732    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9nmd\" (UniqueName: \"kubernetes.io/projected/af900b5e-358a-481c-a359-9844bccc514c-kube-api-access-f9nmd\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.450847 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.450819    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:12:28.451199 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.451167    2561 scope.go:117] "RemoveContainer" containerID="e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9"
Jul 03 14:12:28.451378 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.451359    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cj294_openshift-console-operator(58d23fb7-3fe9-4b16-8719-a155bf7fa66c)\"" pod="openshift-console-operator/console-operator-68b757fb67-cj294" podUID="58d23fb7-3fe9-4b16-8719-a155bf7fa66c"
Jul 03 14:12:28.510824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.510795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.510998 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:28.510887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:28.510998 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.510906    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:12:28.510998 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.510958    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:29.510944922 +0000 UTC m=+51.924841350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : configmap "kube-rbac-proxy" not found
Jul 03 14:12:28.510998 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.510974    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:28.511125 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:28.511012    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:29.511001492 +0000 UTC m=+51.924897924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:29.519131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:29.519101    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:29.519537 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:29.519139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:29.519537 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:29.519236    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:12:29.519537 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:29.519268    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:29.519537 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:29.519289    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:31.519274944 +0000 UTC m=+53.933171372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : configmap "kube-rbac-proxy" not found
Jul 03 14:12:29.519537 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:29.519320    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:31.519307631 +0000 UTC m=+53.933204058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:29.678783 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:29.678762    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-778zb_8e595515-da23-4809-b415-5f5ba0ee0014/dns-node-resolver/0.log"
Jul 03 14:12:30.032443 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.032414    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"]
Jul 03 14:12:30.055433 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.055410    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"]
Jul 03 14:12:30.055555 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.055494    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"
Jul 03 14:12:30.057847 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.057826    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 03 14:12:30.058691 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.058677    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 03 14:12:30.058748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.058707    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vntvm\""
Jul 03 14:12:30.225239 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.225208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:30.225455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.225273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2p8\" (UniqueName: \"kubernetes.io/projected/55a1ea4e-3e19-4b36-a975-778fd47563c7-kube-api-access-dj2p8\") pod \"migrator-d57d58cc9-cb2kc\" (UID: \"55a1ea4e-3e19-4b36-a975-778fd47563c7\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"
Jul 03 14:12:30.225455 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:30.225380    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:12:30.225455 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:30.225429    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls podName:406123bb-f445-4919-b537-dd4c8049c717 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:46.225415685 +0000 UTC m=+68.639312113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls") pod "dns-default-hlpdf" (UID: "406123bb-f445-4919-b537-dd4c8049c717") : secret "dns-default-metrics-tls" not found
Jul 03 14:12:30.325701 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.325629    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2p8\" (UniqueName: \"kubernetes.io/projected/55a1ea4e-3e19-4b36-a975-778fd47563c7-kube-api-access-dj2p8\") pod \"migrator-d57d58cc9-cb2kc\" (UID: \"55a1ea4e-3e19-4b36-a975-778fd47563c7\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"
Jul 03 14:12:30.338428 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.338401    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2p8\" (UniqueName: \"kubernetes.io/projected/55a1ea4e-3e19-4b36-a975-778fd47563c7-kube-api-access-dj2p8\") pod \"migrator-d57d58cc9-cb2kc\" (UID: \"55a1ea4e-3e19-4b36-a975-778fd47563c7\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"
Jul 03 14:12:30.364637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.364613    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"
Jul 03 14:12:30.501531 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.501503    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc"]
Jul 03 14:12:30.504772 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:30.504744    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a1ea4e_3e19_4b36_a975_778fd47563c7.slice/crio-57c553783ff5f493b443ca232110f57296fb90910ce04df74396eb8e390de6ca WatchSource:0}: Error finding container 57c553783ff5f493b443ca232110f57296fb90910ce04df74396eb8e390de6ca: Status 404 returned error can't find the container with id 57c553783ff5f493b443ca232110f57296fb90910ce04df74396eb8e390de6ca
Jul 03 14:12:30.527591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.527571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:30.527879 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:30.527702    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:30.527879 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:30.527718    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:30.527879 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:30.527764    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:38.527749982 +0000 UTC m=+60.941646409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:30.678438 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:30.678416    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4c2sk_80a77bb5-d85b-4df5-b4ca-9f294ead9954/node-ca/0.log"
Jul 03 14:12:31.461033 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:31.460999    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc" event={"ID":"55a1ea4e-3e19-4b36-a975-778fd47563c7","Type":"ContainerStarted","Data":"57c553783ff5f493b443ca232110f57296fb90910ce04df74396eb8e390de6ca"}
Jul 03 14:12:31.536458 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:31.536432    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:31.536878 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:31.536563    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:12:31.536878 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:31.536569    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:31.536878 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:31.536630    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:35.536608631 +0000 UTC m=+57.950505062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : configmap "kube-rbac-proxy" not found
Jul 03 14:12:31.536878 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:31.536716    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:31.536878 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:31.536770    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:35.536757076 +0000 UTC m=+57.950653503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:31.683051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:31.682990    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rmgm9_494eba7d-34c7-493e-8829-84bfaefb21d5/serve-healthcheck-canary/0.log"
Jul 03 14:12:32.468962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.468927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc" event={"ID":"55a1ea4e-3e19-4b36-a975-778fd47563c7","Type":"ContainerStarted","Data":"8c6a2e5d8dd418867267cc69d4b5060a6f246a09336b1f902e70ee27bce15a27"}
Jul 03 14:12:32.469130 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.469011    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc" event={"ID":"55a1ea4e-3e19-4b36-a975-778fd47563c7","Type":"ContainerStarted","Data":"6b82c11bb1aa1ea16f6979eadedd0ee699d9d8f8c7acf69dcddc1e68817ec708"}
Jul 03 14:12:32.490307 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.490233    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cb2kc" podStartSLOduration=0.760915541 podStartE2EDuration="2.490219672s" totalImagesPullingTime="1.729304131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:32.489442884 +0000 UTC m=+54.903339334" watchObservedRunningTime="2026-07-03 14:12:32.490219672 +0000 UTC m=+54.904116121"
Jul 03 14:12:32.589381 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.589351    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:32.589835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.589402    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:32.589835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:32.589813    2561 scope.go:117] "RemoveContainer" containerID="e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9"
Jul 03 14:12:32.590023 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:32.590000    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cj294_openshift-console-operator(58d23fb7-3fe9-4b16-8719-a155bf7fa66c)\"" pod="openshift-console-operator/console-operator-68b757fb67-cj294" podUID="58d23fb7-3fe9-4b16-8719-a155bf7fa66c"
Jul 03 14:12:35.566108 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:35.566078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:35.566108 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:35.566112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:35.566643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:35.566211    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:12:35.566643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:35.566256    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:43.566242535 +0000 UTC m=+65.980138962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : configmap "kube-rbac-proxy" not found
Jul 03 14:12:35.566643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:35.566265    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:35.566643 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:35.566324    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:43.566307487 +0000 UTC m=+65.980203928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:38.586954 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:38.586917    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:38.587370 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:38.587029    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:12:38.587370 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:38.587040    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc78f6d45-mrsnv: secret "image-registry-tls" not found
Jul 03 14:12:38.587370 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:38.587097    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls podName:b2c508bd-bfb4-48bf-9aa5-cdacfa934974 nodeName:}" failed. No retries permitted until 2026-07-03 14:12:54.587082496 +0000 UTC m=+77.000978925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls") pod "image-registry-bc78f6d45-mrsnv" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974") : secret "image-registry-tls" not found
Jul 03 14:12:39.393012 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:39.392983    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rddn"
Jul 03 14:12:43.185163 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.185130    2561 scope.go:117] "RemoveContainer" containerID="e2c6ff6250fc909b78dab76378d4aca0fc88b473b1a671a66eebf4c4284f4ae9"
Jul 03 14:12:43.497678 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.497596    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:12:43.497678 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.497654    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cj294" event={"ID":"58d23fb7-3fe9-4b16-8719-a155bf7fa66c","Type":"ContainerStarted","Data":"2418efa52d8d8f718a093999dec3f869e2cfd6d9396ab66004eec751ab2b4ee5"}
Jul 03 14:12:43.497943 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.497925    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:43.518918 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.518874    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-cj294" podStartSLOduration=18.826440769 podStartE2EDuration="21.518861933s" totalImagesPullingTime="2.692421164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:43.518208368 +0000 UTC m=+65.932104810" watchObservedRunningTime="2026-07-03 14:12:43.518861933 +0000 UTC m=+65.932758374"
Jul 03 14:12:43.623501 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.623477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:43.623652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.623512    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:43.623652 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:43.623610    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:43.623744 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:43.623667    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls podName:af900b5e-358a-481c-a359-9844bccc514c nodeName:}" failed. No retries permitted until 2026-07-03 14:12:59.623652105 +0000 UTC m=+82.037548533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxv2" (UID: "af900b5e-358a-481c-a359-9844bccc514c") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:12:43.623948 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.623933    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af900b5e-358a-481c-a359-9844bccc514c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:43.825335 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.825260    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:43.827495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.827479    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 14:12:43.838556 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.838537    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07fdca58-6749-4863-8fcd-e302905155b2-original-pull-secret\") pod \"global-pull-secret-syncer-dmdfr\" (UID: \"07fdca58-6749-4863-8fcd-e302905155b2\") " pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:43.926148 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.926119    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:43.926302 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.926169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:12:43.928336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.928319    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 14:12:43.928412 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.928326    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 14:12:43.937236 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:43.937217    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 03 14:12:43.937319 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:12:43.937270    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs podName:6c3d773e-c73c-4580-ad51-2843b424cb6e nodeName:}" failed. No retries permitted until 2026-07-03 14:13:47.93725253 +0000 UTC m=+130.351148958 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs") pod "network-metrics-daemon-jphdx" (UID: "6c3d773e-c73c-4580-ad51-2843b424cb6e") : secret "metrics-daemon-secret" not found
Jul 03 14:12:43.939327 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.939307    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 14:12:43.950594 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.950574    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9px\" (UniqueName: \"kubernetes.io/projected/1b329db1-90a9-4bb9-941d-a23411f47c16-kube-api-access-9v9px\") pod \"network-check-target-k4jvq\" (UID: \"1b329db1-90a9-4bb9-941d-a23411f47c16\") " pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:43.993500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:43.993478    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmdfr"
Jul 03 14:12:44.114586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.114560    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m5fkc\""
Jul 03 14:12:44.117983 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.117955    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dmdfr"]
Jul 03 14:12:44.121281 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:44.121260    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07fdca58_6749_4863_8fcd_e302905155b2.slice/crio-ec61e19aab3b6c1649e1361736956ae1c582655212ca05e238dc520efdea7e43 WatchSource:0}: Error finding container ec61e19aab3b6c1649e1361736956ae1c582655212ca05e238dc520efdea7e43: Status 404 returned error can't find the container with id ec61e19aab3b6c1649e1361736956ae1c582655212ca05e238dc520efdea7e43
Jul 03 14:12:44.122127 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.122110    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:44.252578 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.252549    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k4jvq"]
Jul 03 14:12:44.255722 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:44.255694    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b329db1_90a9_4bb9_941d_a23411f47c16.slice/crio-4e8887766d70f6829b1477bb40c8c28c68757e878c14b343eee40dd76a36765d WatchSource:0}: Error finding container 4e8887766d70f6829b1477bb40c8c28c68757e878c14b343eee40dd76a36765d: Status 404 returned error can't find the container with id 4e8887766d70f6829b1477bb40c8c28c68757e878c14b343eee40dd76a36765d
Jul 03 14:12:44.352455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.352431    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-cj294"
Jul 03 14:12:44.502543 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.502459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k4jvq" event={"ID":"1b329db1-90a9-4bb9-941d-a23411f47c16","Type":"ContainerStarted","Data":"4e8887766d70f6829b1477bb40c8c28c68757e878c14b343eee40dd76a36765d"}
Jul 03 14:12:44.503718 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:44.503691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dmdfr" event={"ID":"07fdca58-6749-4863-8fcd-e302905155b2","Type":"ContainerStarted","Data":"ec61e19aab3b6c1649e1361736956ae1c582655212ca05e238dc520efdea7e43"}
Jul 03 14:12:46.246442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:46.246401    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:46.249305 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:46.249254    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/406123bb-f445-4919-b537-dd4c8049c717-metrics-tls\") pod \"dns-default-hlpdf\" (UID: \"406123bb-f445-4919-b537-dd4c8049c717\") " pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:46.483327 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:46.483283    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xht5n\""
Jul 03 14:12:46.491653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:46.491626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:48.853374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:48.853345    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hlpdf"]
Jul 03 14:12:48.856930 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:48.856905    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406123bb_f445_4919_b537_dd4c8049c717.slice/crio-76233fda7290e82db72de05f51228543e68ea8f20a061486a659b3fc52dbdead WatchSource:0}: Error finding container 76233fda7290e82db72de05f51228543e68ea8f20a061486a659b3fc52dbdead: Status 404 returned error can't find the container with id 76233fda7290e82db72de05f51228543e68ea8f20a061486a659b3fc52dbdead
Jul 03 14:12:49.527500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.527459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hlpdf" event={"ID":"406123bb-f445-4919-b537-dd4c8049c717","Type":"ContainerStarted","Data":"76233fda7290e82db72de05f51228543e68ea8f20a061486a659b3fc52dbdead"}
Jul 03 14:12:49.529231 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.529168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k4jvq" event={"ID":"1b329db1-90a9-4bb9-941d-a23411f47c16","Type":"ContainerStarted","Data":"ed1634a23dab8d803dfc8d61164121478d89758e8e546aecf593f0a32db0ff78"}
Jul 03 14:12:49.529384 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.529311    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:12:49.530931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.530904    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dmdfr" event={"ID":"07fdca58-6749-4863-8fcd-e302905155b2","Type":"ContainerStarted","Data":"58addf535f69c5557ecc68b1e8aa3ef52f47606307420ce81d329cd042bd7cbc"}
Jul 03 14:12:49.547356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.547307    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k4jvq" podStartSLOduration=67.083206034 podStartE2EDuration="1m11.547290891s" totalImagesPullingTime="4.464084857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:49.546401417 +0000 UTC m=+71.960297868" watchObservedRunningTime="2026-07-03 14:12:49.547290891 +0000 UTC m=+71.961187343"
Jul 03 14:12:49.565714 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:49.565670    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dmdfr" podStartSLOduration=65.961277035 podStartE2EDuration="1m10.565652984s" totalImagesPullingTime="4.604375949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:49.564954607 +0000 UTC m=+71.978851060" watchObservedRunningTime="2026-07-03 14:12:49.565652984 +0000 UTC m=+71.979549438"
Jul 03 14:12:50.538916 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:50.538882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hlpdf" event={"ID":"406123bb-f445-4919-b537-dd4c8049c717","Type":"ContainerStarted","Data":"8f8973fcc8fa97084649c19d07d4fe4ff3c5dcb380aab421808111d3ac286534"}
Jul 03 14:12:51.544576 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:51.544542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hlpdf" event={"ID":"406123bb-f445-4919-b537-dd4c8049c717","Type":"ContainerStarted","Data":"5d9637818bac5e003236f944e2aae10019cf95cb910e00e6ef984e4e5eb3568d"}
Jul 03 14:12:51.545055 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:51.544678    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:12:51.563465 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:51.563422    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hlpdf" podStartSLOduration=36.017673465 podStartE2EDuration="37.563402084s" totalImagesPullingTime="1.545728619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:51.56241076 +0000 UTC m=+73.976307211" watchObservedRunningTime="2026-07-03 14:12:51.563402084 +0000 UTC m=+73.977298534"
Jul 03 14:12:54.609528 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:54.609489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:54.611940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:54.611913    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"image-registry-bc78f6d45-mrsnv\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") " pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:54.819275 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:54.819241    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkp7q\""
Jul 03 14:12:54.827285 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:54.827269    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:54.959002 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:54.958972    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:12:54.962151 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:54.962124    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c508bd_bfb4_48bf_9aa5_cdacfa934974.slice/crio-00740663ff2e3f31cf8911b40c173e7ce438e62c765f5cacf2c15f0aea5abb86 WatchSource:0}: Error finding container 00740663ff2e3f31cf8911b40c173e7ce438e62c765f5cacf2c15f0aea5abb86: Status 404 returned error can't find the container with id 00740663ff2e3f31cf8911b40c173e7ce438e62c765f5cacf2c15f0aea5abb86
Jul 03 14:12:55.558775 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:55.558743    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" event={"ID":"b2c508bd-bfb4-48bf-9aa5-cdacfa934974","Type":"ContainerStarted","Data":"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"}
Jul 03 14:12:55.558944 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:55.558788    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" event={"ID":"b2c508bd-bfb4-48bf-9aa5-cdacfa934974","Type":"ContainerStarted","Data":"00740663ff2e3f31cf8911b40c173e7ce438e62c765f5cacf2c15f0aea5abb86"}
Jul 03 14:12:55.558944 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:55.558883    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:12:55.580871 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:55.580828    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" podStartSLOduration=33.580814819 podStartE2EDuration="33.580814819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:55.579954704 +0000 UTC m=+77.993851175" watchObservedRunningTime="2026-07-03 14:12:55.580814819 +0000 UTC m=+77.994711292"
Jul 03 14:12:56.411280 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.411242    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:12:56.527994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.527964    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54fcd86df9-nt7sz"]
Jul 03 14:12:56.531036 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.531018    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.545885 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.545864    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54fcd86df9-nt7sz"]
Jul 03 14:12:56.624781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.624751    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-image-registry-private-configuration\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.624898 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.624790    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-ca-trust-extracted\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.624898 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.624833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlktn\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-kube-api-access-xlktn\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.625040 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.625016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-certificates\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.625096 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.625058    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-trusted-ca\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.625200 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.625156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-bound-sa-token\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.625276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.625256    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-tls\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.625332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.625315    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-installation-pull-secrets\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726494    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-image-registry-private-configuration\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-ca-trust-extracted\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726553    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlktn\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-kube-api-access-xlktn\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-certificates\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-trusted-ca\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726655    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-bound-sa-token\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726683    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-tls\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.726808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-installation-pull-secrets\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.727033 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.726962    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-ca-trust-extracted\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.727578 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.727557    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-trusted-ca\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.727861 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.727836    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-certificates\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.729291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.729266    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-image-registry-private-configuration\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.729788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.729769    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-installation-pull-secrets\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.729849 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.729835    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-registry-tls\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.737068 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.737048    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-bound-sa-token\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.737415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.737398    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlktn\" (UniqueName: \"kubernetes.io/projected/9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2-kube-api-access-xlktn\") pod \"image-registry-54fcd86df9-nt7sz\" (UID: \"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2\") " pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.839010 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.838986    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:56.964988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:56.964958    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54fcd86df9-nt7sz"]
Jul 03 14:12:56.967884 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:56.967857    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb6bb63_4bd8_4098_98d3_47fcb1faf5b2.slice/crio-f33a4d954808ebf1d3d076f52e95961eac1169216e45cde7a7393f7910202c7b WatchSource:0}: Error finding container f33a4d954808ebf1d3d076f52e95961eac1169216e45cde7a7393f7910202c7b: Status 404 returned error can't find the container with id f33a4d954808ebf1d3d076f52e95961eac1169216e45cde7a7393f7910202c7b
Jul 03 14:12:57.566280 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:57.566246    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz" event={"ID":"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2","Type":"ContainerStarted","Data":"28c5adf9c8334d7d4eb04aa31e55ac52e89ce0d00e77066bc4ac45f67397e8b3"}
Jul 03 14:12:57.566724 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:57.566297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz" event={"ID":"9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2","Type":"ContainerStarted","Data":"f33a4d954808ebf1d3d076f52e95961eac1169216e45cde7a7393f7910202c7b"}
Jul 03 14:12:57.607126 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:57.607074    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz" podStartSLOduration=1.60705963 podStartE2EDuration="1.60705963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:57.606097889 +0000 UTC m=+80.019994340" watchObservedRunningTime="2026-07-03 14:12:57.60705963 +0000 UTC m=+80.020956080"
Jul 03 14:12:58.571512 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:58.571403    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:12:59.649608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:59.649572    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:59.652023 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:59.651999    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af900b5e-358a-481c-a359-9844bccc514c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxv2\" (UID: \"af900b5e-358a-481c-a359-9844bccc514c\") " pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:59.842088 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:59.842060    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pz6z4\""
Jul 03 14:12:59.850173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:59.850152    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2kxv2"
Jul 03 14:12:59.986074 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:12:59.986048    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxv2"]
Jul 03 14:12:59.989445 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:12:59.989417    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf900b5e_358a_481c_a359_9844bccc514c.slice/crio-1d8cf4d6fde871ed0a54b5e2959c50e1f60a510810a8d9608e1330ccfd42d62e WatchSource:0}: Error finding container 1d8cf4d6fde871ed0a54b5e2959c50e1f60a510810a8d9608e1330ccfd42d62e: Status 404 returned error can't find the container with id 1d8cf4d6fde871ed0a54b5e2959c50e1f60a510810a8d9608e1330ccfd42d62e
Jul 03 14:13:00.580721 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:00.580679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxv2" event={"ID":"af900b5e-358a-481c-a359-9844bccc514c","Type":"ContainerStarted","Data":"1d8cf4d6fde871ed0a54b5e2959c50e1f60a510810a8d9608e1330ccfd42d62e"}
Jul 03 14:13:01.550032 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:01.550000    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hlpdf"
Jul 03 14:13:04.597304 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:04.597275    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxv2" event={"ID":"af900b5e-358a-481c-a359-9844bccc514c","Type":"ContainerStarted","Data":"6a6370a087fe6537df71db2ee04d059f78c01e998d879f1988ebb3d32c94d28e"}
Jul 03 14:13:04.597687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:04.597327    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxv2" event={"ID":"af900b5e-358a-481c-a359-9844bccc514c","Type":"ContainerStarted","Data":"1999af7d8e48d688d1a65f0fe6cfa729bbd4b9ada69cb0f9f5a1544e736612f1"}
Jul 03 14:13:07.608816 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:07.608777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxv2" event={"ID":"af900b5e-358a-481c-a359-9844bccc514c","Type":"ContainerStarted","Data":"50ee0cdf3954d7c46f992004b461ab4b19499a158181b1a85e2283bf2b9b3a5a"}
Jul 03 14:13:07.628864 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:07.628818    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2kxv2" podStartSLOduration=33.885157871 podStartE2EDuration="40.628804129s" totalImagesPullingTime="6.743646258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:12:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:07.627706841 +0000 UTC m=+90.041603292" watchObservedRunningTime="2026-07-03 14:13:07.628804129 +0000 UTC m=+90.042700579"
Jul 03 14:13:17.572317 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:17.572286    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:13:19.324200 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.324153    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8sbkm"]
Jul 03 14:13:19.328535 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.328511    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.331466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331445    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 03 14:13:19.331573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331515    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 03 14:13:19.331573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331530    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 03 14:13:19.331684 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331587    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l7pjz\""
Jul 03 14:13:19.331684 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331654    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 03 14:13:19.331960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.331940    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 03 14:13:19.332457 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.332436    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 03 14:13:19.403589 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403566    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-textfile\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403593    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-metrics-client-ca\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403616    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj6ql\" (UniqueName: \"kubernetes.io/projected/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-kube-api-access-lj6ql\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-root\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-sys\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403685    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403724    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403743    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.403888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.403781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-wtmp\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-wtmp\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-textfile\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504313    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-metrics-client-ca\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj6ql\" (UniqueName: \"kubernetes.io/projected/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-kube-api-access-lj6ql\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504384    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-root\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504404    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-wtmp\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504407    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-sys\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504461    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-root\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504467    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-sys\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504468    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.504926 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.504903    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-textfile\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.505014 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:13:19.504930    2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 03 14:13:19.505014 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:13:19.504997    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls podName:00137202-1f2e-4cea-bebf-f5c00e0ac8ea nodeName:}" failed. No retries permitted until 2026-07-03 14:13:20.004979452 +0000 UTC m=+102.418875880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls") pod "node-exporter-8sbkm" (UID: "00137202-1f2e-4cea-bebf-f5c00e0ac8ea") : secret "node-exporter-tls" not found
Jul 03 14:13:19.505283 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.505254    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-metrics-client-ca\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.505936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.505905    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.507669 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.507642    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.518412 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.518394    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj6ql\" (UniqueName: \"kubernetes.io/projected/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-kube-api-access-lj6ql\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:19.578817 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:19.578771    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54fcd86df9-nt7sz"
Jul 03 14:13:20.009312 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:20.009270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:20.011638 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:20.011609    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00137202-1f2e-4cea-bebf-f5c00e0ac8ea-node-exporter-tls\") pod \"node-exporter-8sbkm\" (UID: \"00137202-1f2e-4cea-bebf-f5c00e0ac8ea\") " pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:20.237758 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:20.237724    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sbkm"
Jul 03 14:13:20.248783 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:20.248755    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00137202_1f2e_4cea_bebf_f5c00e0ac8ea.slice/crio-6f9777a858f351f34201b5980e2675341c357060a78a477cfe1111136ed857f6 WatchSource:0}: Error finding container 6f9777a858f351f34201b5980e2675341c357060a78a477cfe1111136ed857f6: Status 404 returned error can't find the container with id 6f9777a858f351f34201b5980e2675341c357060a78a477cfe1111136ed857f6
Jul 03 14:13:20.541435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:20.541411    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k4jvq"
Jul 03 14:13:20.652921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:20.652891    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sbkm" event={"ID":"00137202-1f2e-4cea-bebf-f5c00e0ac8ea","Type":"ContainerStarted","Data":"6f9777a858f351f34201b5980e2675341c357060a78a477cfe1111136ed857f6"}
Jul 03 14:13:21.355923 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.355894    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"]
Jul 03 14:13:21.367634 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.367614    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.373795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.373428    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 03 14:13:21.373795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.373612    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 03 14:13:21.373795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.373747    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 03 14:13:21.374059 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.373927    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2dvts\""
Jul 03 14:13:21.374170 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.374152    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 03 14:13:21.374260 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.374219    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"]
Jul 03 14:13:21.374466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.374415    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-85kv17vb5u9j\""
Jul 03 14:13:21.374867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.374575    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 03 14:13:21.523204 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523338 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523219    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523338 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523282    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-grpc-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523338 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523343    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523360    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523378    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/734808b7-e7c0-438b-a60b-b7568f784d9b-metrics-client-ca\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.523464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.523416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpvn\" (UniqueName: \"kubernetes.io/projected/734808b7-e7c0-438b-a60b-b7568f784d9b-kube-api-access-fjpvn\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624318 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624294    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624365    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/734808b7-e7c0-438b-a60b-b7568f784d9b-metrics-client-ca\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624544    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpvn\" (UniqueName: \"kubernetes.io/projected/734808b7-e7c0-438b-a60b-b7568f784d9b-kube-api-access-fjpvn\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624648 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624639    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.624987 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.624689    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-grpc-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.625352 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.625274    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/734808b7-e7c0-438b-a60b-b7568f784d9b-metrics-client-ca\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.626939 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.626913    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.627033 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.626973    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.627310 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.627291    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.627441 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.627426    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.627655 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.627638    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.627691 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.627667    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/734808b7-e7c0-438b-a60b-b7568f784d9b-secret-grpc-tls\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.633366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.633351    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpvn\" (UniqueName: \"kubernetes.io/projected/734808b7-e7c0-438b-a60b-b7568f784d9b-kube-api-access-fjpvn\") pod \"thanos-querier-657b7cbf4b-hk7fz\" (UID: \"734808b7-e7c0-438b-a60b-b7568f784d9b\") " pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.677787 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.677765    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:21.835515 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:21.835491    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"]
Jul 03 14:13:21.884050 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:21.883984    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734808b7_e7c0_438b_a60b_b7568f784d9b.slice/crio-3ea91db57e0d6605a78a7e989b18af5cff8d55078e17c0ed90478da20f8f1f37 WatchSource:0}: Error finding container 3ea91db57e0d6605a78a7e989b18af5cff8d55078e17c0ed90478da20f8f1f37: Status 404 returned error can't find the container with id 3ea91db57e0d6605a78a7e989b18af5cff8d55078e17c0ed90478da20f8f1f37
Jul 03 14:13:22.586677 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:22.586615    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" podUID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" containerName="registry" containerID="cri-o://9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3" gracePeriod=30
Jul 03 14:13:22.662231 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:22.662169    2561 generic.go:356] "Generic (PLEG): container finished" podID="00137202-1f2e-4cea-bebf-f5c00e0ac8ea" containerID="752752e418a98265d7d3087b78b0042b191c04619e886ab622d27aa520041b3f" exitCode=0
Jul 03 14:13:22.662608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:22.662282    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sbkm" event={"ID":"00137202-1f2e-4cea-bebf-f5c00e0ac8ea","Type":"ContainerDied","Data":"752752e418a98265d7d3087b78b0042b191c04619e886ab622d27aa520041b3f"}
Jul 03 14:13:22.663884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:22.663852    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"3ea91db57e0d6605a78a7e989b18af5cff8d55078e17c0ed90478da20f8f1f37"}
Jul 03 14:13:22.897842 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:22.897820    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:13:23.036005 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.035978    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036017    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036062    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036095    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036124    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036159    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7rsd\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036271    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036297    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca\") pod \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\" (UID: \"b2c508bd-bfb4-48bf-9aa5-cdacfa934974\") "
Jul 03 14:13:23.036920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036460    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:13:23.036920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036523    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-certificates\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.036920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.036899    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:13:23.039586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.039538    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:13:23.039586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.039557    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:13:23.039586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.039557    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd" (OuterVolumeSpecName: "kube-api-access-b7rsd") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "kube-api-access-b7rsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:13:23.039805 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.039768    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:13:23.040001 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.039844    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:13:23.047759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.047731    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b2c508bd-bfb4-48bf-9aa5-cdacfa934974" (UID: "b2c508bd-bfb4-48bf-9aa5-cdacfa934974"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.137923    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-bound-sa-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.137954    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-trusted-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.137968    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-ca-trust-extracted\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.137985    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-image-registry-private-configuration\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.138000    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.138014    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-installation-pull-secrets\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.138439 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.138028    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7rsd\" (UniqueName: \"kubernetes.io/projected/b2c508bd-bfb4-48bf-9aa5-cdacfa934974-kube-api-access-b7rsd\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:13:23.669925 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.669865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sbkm" event={"ID":"00137202-1f2e-4cea-bebf-f5c00e0ac8ea","Type":"ContainerStarted","Data":"1fdb01f8f6a2392e9fac06a9e5f60446e111e0d57bea0c85e3159da54347abbc"}
Jul 03 14:13:23.670321 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.669949    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sbkm" event={"ID":"00137202-1f2e-4cea-bebf-f5c00e0ac8ea","Type":"ContainerStarted","Data":"eb47ff34c69a580cb13cd207cd76848e0dc25d7cc7050e27c64340d64c3ee600"}
Jul 03 14:13:23.671356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.671329    2561 generic.go:356] "Generic (PLEG): container finished" podID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" containerID="9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3" exitCode=0
Jul 03 14:13:23.671482 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.671375    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" event={"ID":"b2c508bd-bfb4-48bf-9aa5-cdacfa934974","Type":"ContainerDied","Data":"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"}
Jul 03 14:13:23.671482 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.671383    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv"
Jul 03 14:13:23.671482 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.671403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc78f6d45-mrsnv" event={"ID":"b2c508bd-bfb4-48bf-9aa5-cdacfa934974","Type":"ContainerDied","Data":"00740663ff2e3f31cf8911b40c173e7ce438e62c765f5cacf2c15f0aea5abb86"}
Jul 03 14:13:23.671482 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.671430    2561 scope.go:117] "RemoveContainer" containerID="9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"
Jul 03 14:13:23.692470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.692407    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8sbkm" podStartSLOduration=3.260616047 podStartE2EDuration="4.692392303s" totalImagesPullingTime="1.329818738s" totalInitContainerRuntime="101.957518ms" isStatefulPod=true podCreationTimestamp="2026-07-03 14:13:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:23.691828433 +0000 UTC m=+106.105724883" watchObservedRunningTime="2026-07-03 14:13:23.692392303 +0000 UTC m=+106.106288752"
Jul 03 14:13:23.711518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.711498    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:13:23.720533 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.720495    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bc78f6d45-mrsnv"]
Jul 03 14:13:23.839111 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.839089    2561 scope.go:117] "RemoveContainer" containerID="9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"
Jul 03 14:13:23.839504 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:13:23.839478    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3\": container with ID starting with 9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3 not found: ID does not exist" containerID="9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"
Jul 03 14:13:23.839604 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.839519    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3"} err="failed to get container status \"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3\": rpc error: code = NotFound desc = could not find container \"9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3\": container with ID starting with 9d27869ac2bc0307149f277245e74b83de8e296dca79b2f021f4d58d5bbfc9f3 not found: ID does not exist"
Jul 03 14:13:23.863622 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.863600    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d9546c774-4rpkx"]
Jul 03 14:13:23.863849 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.863837    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" containerName="registry"
Jul 03 14:13:23.863888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.863851    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" containerName="registry"
Jul 03 14:13:23.863917 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.863903    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" containerName="registry"
Jul 03 14:13:23.876655 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.876634    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d9546c774-4rpkx"]
Jul 03 14:13:23.876723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.876710    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:23.879086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879059    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 03 14:13:23.879356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879335    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 03 14:13:23.879462 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879342    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e9q386i7ct53a\""
Jul 03 14:13:23.879517 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879456    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vglh4\""
Jul 03 14:13:23.879595 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879580    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 03 14:13:23.879714 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:23.879692    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 03 14:13:24.046921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.046842    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.046921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.046876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82f7d731-45fd-44f8-8eb2-e638d58bcab5-audit-log\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.047128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.046976    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9qj6\" (UniqueName: \"kubernetes.io/projected/82f7d731-45fd-44f8-8eb2-e638d58bcab5-kube-api-access-n9qj6\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.047128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.047067    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-metrics-server-audit-profiles\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.047128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.047111    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-tls\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.047299 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.047149    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-client-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.047299 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.047206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-client-certs\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.133365 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.133335    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"]
Jul 03 14:13:24.147549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147519    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.147695 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147658    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-metrics-server-audit-profiles\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.147695 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147692    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-tls\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.147798 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-client-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.147798 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147738    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-client-certs\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.147935 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.147913    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.148426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.148009    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82f7d731-45fd-44f8-8eb2-e638d58bcab5-audit-log\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.148426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.148085    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9qj6\" (UniqueName: \"kubernetes.io/projected/82f7d731-45fd-44f8-8eb2-e638d58bcab5-kube-api-access-n9qj6\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.148588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.148452    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82f7d731-45fd-44f8-8eb2-e638d58bcab5-audit-log\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.148879 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.148858    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.148988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.148966    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82f7d731-45fd-44f8-8eb2-e638d58bcab5-metrics-server-audit-profiles\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.150538 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.150518    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-client-ca-bundle\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.150650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.150519    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-client-certs\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.150650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.150585    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82f7d731-45fd-44f8-8eb2-e638d58bcab5-secret-metrics-server-tls\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.151705 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.151689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5k82n\""
Jul 03 14:13:24.151974 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.151959    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 03 14:13:24.162840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.162820    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9qj6\" (UniqueName: \"kubernetes.io/projected/82f7d731-45fd-44f8-8eb2-e638d58bcab5-kube-api-access-n9qj6\") pod \"metrics-server-d9546c774-4rpkx\" (UID: \"82f7d731-45fd-44f8-8eb2-e638d58bcab5\") " pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.167057 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.167035    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"]
Jul 03 14:13:24.186062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.186044    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:24.189072 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.189052    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c508bd-bfb4-48bf-9aa5-cdacfa934974" path="/var/lib/kubelet/pods/b2c508bd-bfb4-48bf-9aa5-cdacfa934974/volumes"
Jul 03 14:13:24.249045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.249015    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.249143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.249103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfzh\" (UniqueName: \"kubernetes.io/projected/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-kube-api-access-cwfzh\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.341081 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.340971    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d9546c774-4rpkx"]
Jul 03 14:13:24.344551 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:24.343763    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f7d731_45fd_44f8_8eb2_e638d58bcab5.slice/crio-82f528928751f4a23c35d9226f3467319d1c60c87b824c47c9d12bc0ecd3ca3f WatchSource:0}: Error finding container 82f528928751f4a23c35d9226f3467319d1c60c87b824c47c9d12bc0ecd3ca3f: Status 404 returned error can't find the container with id 82f528928751f4a23c35d9226f3467319d1c60c87b824c47c9d12bc0ecd3ca3f
Jul 03 14:13:24.350457 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.350427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.350545 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.350473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfzh\" (UniqueName: \"kubernetes.io/projected/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-kube-api-access-cwfzh\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.352988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.352964    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.362054 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.362038    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfzh\" (UniqueName: \"kubernetes.io/projected/f5da2aa3-df10-43c5-8c4f-0d945a8dffc3-kube-api-access-cwfzh\") pod \"monitoring-plugin-d5dcff7cf-tzqrn\" (UID: \"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.462917 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.462895    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:24.602027 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.602001    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"]
Jul 03 14:13:24.605040 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:24.605012    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5da2aa3_df10_43c5_8c4f_0d945a8dffc3.slice/crio-310b4b61d0b98ea33651cd6a77e5801355536a7f1b56df90feb62e4e0cb9904d WatchSource:0}: Error finding container 310b4b61d0b98ea33651cd6a77e5801355536a7f1b56df90feb62e4e0cb9904d: Status 404 returned error can't find the container with id 310b4b61d0b98ea33651cd6a77e5801355536a7f1b56df90feb62e4e0cb9904d
Jul 03 14:13:24.677546 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.677510    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"55700120feee661be9fa5836d8378c84005931c17b904bf2160ba70eed4f540b"}
Jul 03 14:13:24.677908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.677578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"ffb0b633ce38d0ba93c8a4341883880ddaf46173827ad7282cf32b4f7cbe204f"}
Jul 03 14:13:24.677908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.677605    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"064a8d8bdae6385b04f99ee69f83043bc38049ac725dbf80c3704578274c22da"}
Jul 03 14:13:24.678777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.678756    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn" event={"ID":"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3","Type":"ContainerStarted","Data":"310b4b61d0b98ea33651cd6a77e5801355536a7f1b56df90feb62e4e0cb9904d"}
Jul 03 14:13:24.679897 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:24.679881    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx" event={"ID":"82f7d731-45fd-44f8-8eb2-e638d58bcab5","Type":"ContainerStarted","Data":"82f528928751f4a23c35d9226f3467319d1c60c87b824c47c9d12bc0ecd3ca3f"}
Jul 03 14:13:25.688586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.688548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"7eea9892916b41769fa7d78d2d2dd1bed2d472757a9bb7c857d2f916370ee383"}
Jul 03 14:13:25.689024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.688626    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"cae970cad3a3c03732760bf69c4cf070982006be76fba996607cd45a6d07cdec"}
Jul 03 14:13:25.689024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.688660    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" event={"ID":"734808b7-e7c0-438b-a60b-b7568f784d9b","Type":"ContainerStarted","Data":"d0393fe50ecfa67d8d93013001f6913fd7b6912981b0c6b313fd928de0244b81"}
Jul 03 14:13:25.689024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.688787    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:25.699028 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.699000    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:13:25.702158 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.702136    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.704746 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.704729    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 03 14:13:25.705052 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.704727    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 03 14:13:25.705052 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.704949    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 03 14:13:25.705233 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705148    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5c2l2hvki51ns\""
Jul 03 14:13:25.705233 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705216    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 03 14:13:25.705791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705360    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 03 14:13:25.705791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705649    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 03 14:13:25.705791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705666    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42stv\""
Jul 03 14:13:25.705791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705740    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 03 14:13:25.705791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705770    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 03 14:13:25.706047 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705914    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 03 14:13:25.706047 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.705990    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 03 14:13:25.706141 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.706106    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 03 14:13:25.706247 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.706228    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 03 14:13:25.706347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.706333    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 03 14:13:25.708523 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.708503    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 03 14:13:25.713729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.713685    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz" podStartSLOduration=1.560743494 podStartE2EDuration="4.713666833s" totalImagesPullingTime="3.152923339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:13:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:25.712497307 +0000 UTC m=+108.126393756" watchObservedRunningTime="2026-07-03 14:13:25.713666833 +0000 UTC m=+108.127563285"
Jul 03 14:13:25.720948 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.720922    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:13:25.864194 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864147    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864375 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864214    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864375 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864491 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864376    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864491 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864436    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864577 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864502    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864577 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864649 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864576    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864649 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864605    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864649 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864634    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864657    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864676    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864779    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.864850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.864804    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj225\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865404 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865326    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865404 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865388    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865497 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865428    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865497 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865462    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.865497 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.865492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.966744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.966744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.966744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj225\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966858    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966894    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966925    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966957    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.966986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967364 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.967008    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.967364 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.967074    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968114    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968273 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968223    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968348 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968348 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968449 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968449 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968449 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968628 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968628 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968628 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968628 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968596    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.968963 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.968936    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.969097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.969064    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.969590 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.969564    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.970631 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.970601    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.970952 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.970929    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.971243 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.971201    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.971243 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.971218    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.972591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.972256    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.972591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.972332    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.972591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.972551    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.972794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.972635    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.973912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.973887    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.973991 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.973893    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.974116 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.974095    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.974343 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.974319    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.975366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.975348    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.976553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.976528    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj225\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:25.977562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:25.977542    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:26.015031 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.014999    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:26.593607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.593583    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:13:26.597699 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:26.597663    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde392140_486c_4d79_beea_618ed0b840e1.slice/crio-9e11c2f6d4e3c2452fec511134bd43da833583febb3e46dedf9995ebec6603b8 WatchSource:0}: Error finding container 9e11c2f6d4e3c2452fec511134bd43da833583febb3e46dedf9995ebec6603b8: Status 404 returned error can't find the container with id 9e11c2f6d4e3c2452fec511134bd43da833583febb3e46dedf9995ebec6603b8
Jul 03 14:13:26.693752 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.693670    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn" event={"ID":"f5da2aa3-df10-43c5-8c4f-0d945a8dffc3","Type":"ContainerStarted","Data":"c70512a2e15f1d4d66208fd211e1f1776c14bcc015c9c0ff19d760d1832133d2"}
Jul 03 14:13:26.694152 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.693878    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:26.695132 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.695109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"9e11c2f6d4e3c2452fec511134bd43da833583febb3e46dedf9995ebec6603b8"}
Jul 03 14:13:26.696827 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.696802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx" event={"ID":"82f7d731-45fd-44f8-8eb2-e638d58bcab5","Type":"ContainerStarted","Data":"71b3771bc2bc4cc1ac24b110ae3a5772e4cb5f42bb8b4c3b798a18ddcd39d612"}
Jul 03 14:13:26.700261 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.700243    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn"
Jul 03 14:13:26.736031 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.735984    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tzqrn" podStartSLOduration=0.894308589 podStartE2EDuration="2.735967751s" totalImagesPullingTime="1.841659162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:26.718096241 +0000 UTC m=+109.131992703" watchObservedRunningTime="2026-07-03 14:13:26.735967751 +0000 UTC m=+109.149864204"
Jul 03 14:13:26.758207 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:26.758143    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx" podStartSLOduration=1.658500374 podStartE2EDuration="3.75812531s" totalImagesPullingTime="2.099624936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:26.757006262 +0000 UTC m=+109.170902713" watchObservedRunningTime="2026-07-03 14:13:26.75812531 +0000 UTC m=+109.172021762"
Jul 03 14:13:28.705867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:28.705830    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790" exitCode=0
Jul 03 14:13:28.706379 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:28.705918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"}
Jul 03 14:13:31.704550 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:31.704522    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-657b7cbf4b-hk7fz"
Jul 03 14:13:32.726115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726075    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"}
Jul 03 14:13:32.726518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"}
Jul 03 14:13:32.726518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726212    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"}
Jul 03 14:13:32.726518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726238    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"}
Jul 03 14:13:32.726518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726261    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"}
Jul 03 14:13:32.726518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.726285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerStarted","Data":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"}
Jul 03 14:13:32.758292 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:32.758221    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.338867294 podStartE2EDuration="7.758202544s" totalImagesPullingTime="4.396863695s" totalInitContainerRuntime="22.471555ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:13:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:32.756376838 +0000 UTC m=+115.170273290" watchObservedRunningTime="2026-07-03 14:13:32.758202544 +0000 UTC m=+115.172098995"
Jul 03 14:13:36.015152 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:36.015120    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:44.188696 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:44.188665    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:44.189064 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:44.188718    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:13:47.955296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:47.955254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:13:47.957833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:47.957808    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c3d773e-c73c-4580-ad51-2843b424cb6e-metrics-certs\") pod \"network-metrics-daemon-jphdx\" (UID: \"6c3d773e-c73c-4580-ad51-2843b424cb6e\") " pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:13:48.005254 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:48.005224    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kg466\""
Jul 03 14:13:48.013022 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:48.012997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jphdx"
Jul 03 14:13:48.163650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:48.163613    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jphdx"]
Jul 03 14:13:48.166823 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:13:48.166793    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3d773e_c73c_4580_ad51_2843b424cb6e.slice/crio-f668d25180c1124140adf0ed3c92f75f9d235094e1f39fcb218ac7ddaa45037b WatchSource:0}: Error finding container f668d25180c1124140adf0ed3c92f75f9d235094e1f39fcb218ac7ddaa45037b: Status 404 returned error can't find the container with id f668d25180c1124140adf0ed3c92f75f9d235094e1f39fcb218ac7ddaa45037b
Jul 03 14:13:48.788457 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:48.788422    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jphdx" event={"ID":"6c3d773e-c73c-4580-ad51-2843b424cb6e","Type":"ContainerStarted","Data":"f668d25180c1124140adf0ed3c92f75f9d235094e1f39fcb218ac7ddaa45037b"}
Jul 03 14:13:49.793960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:49.793919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jphdx" event={"ID":"6c3d773e-c73c-4580-ad51-2843b424cb6e","Type":"ContainerStarted","Data":"42fa3eedf76f0931c54ed922db9a6cc98c94d19ed361f2af224f207e9d450c92"}
Jul 03 14:13:49.794444 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:49.793979    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jphdx" event={"ID":"6c3d773e-c73c-4580-ad51-2843b424cb6e","Type":"ContainerStarted","Data":"fe08c10f57c69a7d5a7b82d5cd1fa676460893f52b54044703400a3dbc1e955b"}
Jul 03 14:13:49.813370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:13:49.813321    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jphdx" podStartSLOduration=130.922637465 podStartE2EDuration="2m11.81330579s" totalImagesPullingTime="890.668325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:13:49.812346458 +0000 UTC m=+132.226242907" watchObservedRunningTime="2026-07-03 14:13:49.81330579 +0000 UTC m=+132.227202239"
Jul 03 14:14:04.191759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:04.191734    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:14:04.195498 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:04.195453    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d9546c774-4rpkx"
Jul 03 14:14:26.015805 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:26.015766    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:26.035860 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:26.035833    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:26.948744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:26.948718    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:44.128708 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.128678    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.129006    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="prometheus" containerID="cri-o://a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce" gracePeriod=600
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.129020    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="config-reloader" containerID="cri-o://cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde" gracePeriod=600
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.129067    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1" gracePeriod=600
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.129047    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy" containerID="cri-o://833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170" gracePeriod=600
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.129046    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="thanos-sidecar" containerID="cri-o://5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c" gracePeriod=600
Jul 03 14:14:44.129284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.128994    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-web" containerID="cri-o://3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb" gracePeriod=600
Jul 03 14:14:44.368372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.368350    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:44.410205 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410127    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410205 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410162    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410210    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410238    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410267    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410292    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410318    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410344    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410402    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410427    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410466    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410509    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410518    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410550    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410574    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410598    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410638    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410643    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.410734 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410694    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410741    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410753    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj225\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225\") pod \"de392140-486c-4d79-beea-618ed0b840e1\" (UID: \"de392140-486c-4d79-beea-618ed0b840e1\") "
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.410994    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.411011    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.411034    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-metrics-client-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.411340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.411037    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.412858 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.412482    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.412858 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.412615    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:14:44.413617 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.413587    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.414305 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.413973    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.414305 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.414038    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.414570 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.414505    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.414570 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.414547    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:14:44.416011 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.415976    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:14:44.416105 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.416052    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.416882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.416371    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.416882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.416397    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config" (OuterVolumeSpecName: "config") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.416882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.416482    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.417091 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.416919    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225" (OuterVolumeSpecName: "kube-api-access-vj225") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "kube-api-access-vj225". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:14:44.417284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.417245    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.417962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.417923    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out" (OuterVolumeSpecName: "config-out") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:14:44.426910 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.426886    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config" (OuterVolumeSpecName: "web-config") pod "de392140-486c-4d79-beea-618ed0b840e1" (UID: "de392140-486c-4d79-beea-618ed0b840e1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:14:44.511642 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511620    2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-config\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511642 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511640    2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-web-config\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511649    2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-config-out\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511657    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511668    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511678    2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511687    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511695    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj225\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-kube-api-access-vj225\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511704    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511712    2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-kube-rbac-proxy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511720    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-db\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511729    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511738    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511748    2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-metrics-client-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511758    2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de392140-486c-4d79-beea-618ed0b840e1-secret-grpc-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511765    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de392140-486c-4d79-beea-618ed0b840e1-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:44.511771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:44.511774    2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de392140-486c-4d79-beea-618ed0b840e1-tls-assets\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:14:45.001835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001793    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1" exitCode=0
Jul 03 14:14:45.001835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001823    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170" exitCode=0
Jul 03 14:14:45.001835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001830    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb" exitCode=0
Jul 03 14:14:45.001835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001837    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c" exitCode=0
Jul 03 14:14:45.001835 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001843    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde" exitCode=0
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001848    2561 generic.go:356] "Generic (PLEG): container finished" podID="de392140-486c-4d79-beea-618ed0b840e1" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce" exitCode=0
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"}
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001936    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001953    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"}
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.001982    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"}
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.002010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"}
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.002030    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.002049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"}
Jul 03 14:14:45.002112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.002089    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"}
Jul 03 14:14:45.002495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.002130    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de392140-486c-4d79-beea-618ed0b840e1","Type":"ContainerDied","Data":"9e11c2f6d4e3c2452fec511134bd43da833583febb3e46dedf9995ebec6603b8"}
Jul 03 14:14:45.011051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.011023    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.019024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.019009    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.025558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.025535    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.031922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.031898    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:45.033813 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.033794    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.034588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.034570    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:45.040718 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.040703    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.047722 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.047705    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.054022 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054008    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.054384 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.054364    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.054455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054391    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.054455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054409    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.054639 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.054620    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.054707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054650    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.054707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054674    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.054927 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.054908    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.054966 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054933    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.054966 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.054949    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.055170 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.055154    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.055241 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055175    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.055241 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055209    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.055439 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.055423    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.055500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055447    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.055500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055469    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.055697 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.055680    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.055738 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055702    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.055738 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055716    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.055910 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:14:45.055894    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.055946 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055914    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.055946 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.055926    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.056127 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056105    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.056171 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056131    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.056347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056329    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.056415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056348    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.056589 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056574    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.056635 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056589    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.056771 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056757    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.056814 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056773    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.056996 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056980    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.057036 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.056997    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.057292 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057275    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.057335 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057293    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.057504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057487    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.057582 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057506    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.057718 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057700    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.057767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057719    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.057936 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057918    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.058001 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.057938    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.058131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058115    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.058215 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058132    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.058347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058331    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.058416 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058348    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.058553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058535    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.058600 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058556    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.058729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058712    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.058772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058732    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.058943 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058926    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.059008 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.058943    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.059163 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059147    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.059163 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059163    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.059394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059364    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.059394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059384    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.059590 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059573    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.059635 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059591    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.059794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059778    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.059859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059795    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.059982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059964    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.060030 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.059983    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.060210 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060191    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.060281 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060211    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.060404 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060387    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.060468 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060405    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.060641 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060622    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.060685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060642    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.060848 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060829    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.060930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.060849    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.061078 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061051    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.061078 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061070    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.061270 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061253    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.061270 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061270    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.061475 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061458    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.061538 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061477    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.061680 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061664    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.061744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061681    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.061900 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061883    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.061964 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.061900    2561 scope.go:117] "RemoveContainer" containerID="b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"
Jul 03 14:14:45.062111 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062096    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1"} err="failed to get container status \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": rpc error: code = NotFound desc = could not find container \"b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1\": container with ID starting with b7dadd81ef5ca1d8ec5103209029aacff1f30bea5085079e81b85c04551c42f1 not found: ID does not exist"
Jul 03 14:14:45.062155 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062112    2561 scope.go:117] "RemoveContainer" containerID="833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"
Jul 03 14:14:45.062314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062296    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170"} err="failed to get container status \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": rpc error: code = NotFound desc = could not find container \"833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170\": container with ID starting with 833672e6c561122180eed1c570d0bb5f4d879596d5f418e839ad4d8bc2a5a170 not found: ID does not exist"
Jul 03 14:14:45.062314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062314    2561 scope.go:117] "RemoveContainer" containerID="3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"
Jul 03 14:14:45.062511 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062490    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb"} err="failed to get container status \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": rpc error: code = NotFound desc = could not find container \"3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb\": container with ID starting with 3fde880df503b763495b3d24fa518d5b283fb6e99b7d3a5a481e199a04db52fb not found: ID does not exist"
Jul 03 14:14:45.062577 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062513    2561 scope.go:117] "RemoveContainer" containerID="5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"
Jul 03 14:14:45.062741 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062725    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c"} err="failed to get container status \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": rpc error: code = NotFound desc = could not find container \"5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c\": container with ID starting with 5c49ec6336b1df52c437ac256970bfcaa4bee5f3fa6d35b1a3c2f4745eaed80c not found: ID does not exist"
Jul 03 14:14:45.062785 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062741    2561 scope.go:117] "RemoveContainer" containerID="cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"
Jul 03 14:14:45.062907 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062889    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde"} err="failed to get container status \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": rpc error: code = NotFound desc = could not find container \"cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde\": container with ID starting with cf380a3537ce71fbb0698f8314f5761b2034970c8ce2f98b97c40af09cd16fde not found: ID does not exist"
Jul 03 14:14:45.062994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.062909    2561 scope.go:117] "RemoveContainer" containerID="a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"
Jul 03 14:14:45.063064 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.063043    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce"} err="failed to get container status \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": rpc error: code = NotFound desc = could not find container \"a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce\": container with ID starting with a1e0b7c4ed00be6641cddc51cb92f5fabe54c6af675d74fbbdd273eb394201ce not found: ID does not exist"
Jul 03 14:14:45.063104 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.063065    2561 scope.go:117] "RemoveContainer" containerID="106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"
Jul 03 14:14:45.063491 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.063374    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790"} err="failed to get container status \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": rpc error: code = NotFound desc = could not find container \"106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790\": container with ID starting with 106d28862dde55de61a5359fc3c61baa27f779da88f5d899c2543381fd570790 not found: ID does not exist"
Jul 03 14:14:45.064752 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.064730    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:45.065073 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065057    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065077    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065088    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-web"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065094    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-web"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065100    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-thanos"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065105    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-thanos"
Jul 03 14:14:45.065112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065113    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="prometheus"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065119    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="prometheus"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065126    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="thanos-sidecar"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065131    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="thanos-sidecar"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065137    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="init-config-reloader"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065142    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="init-config-reloader"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065147    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="config-reloader"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065152    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="config-reloader"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065226    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-web"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065239    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065249    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="kube-rbac-proxy-thanos"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065263    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="prometheus"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065272    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="thanos-sidecar"
Jul 03 14:14:45.065301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.065280    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de392140-486c-4d79-beea-618ed0b840e1" containerName="config-reloader"
Jul 03 14:14:45.070220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.070167    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.073114 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073078    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 03 14:14:45.073262 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073242    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 03 14:14:45.073373 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073316    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 03 14:14:45.073542 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073514    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 03 14:14:45.073662 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073569    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 03 14:14:45.073755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073748    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42stv\""
Jul 03 14:14:45.073825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073759    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5c2l2hvki51ns\""
Jul 03 14:14:45.073825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073786    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 03 14:14:45.073825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073786    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 03 14:14:45.073964 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.073833    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 03 14:14:45.074553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.074529    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 03 14:14:45.074614 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.074539    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 03 14:14:45.074986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.074969    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 03 14:14:45.075548 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.075530    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 03 14:14:45.076756 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.076738    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 03 14:14:45.079656 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.079636    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 03 14:14:45.087707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.087688    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:45.115901 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.115879    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.115999 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.115912    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.115999 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.115934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.115999 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.115977    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116002    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116032    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116079    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116120    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hvxm\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-kube-api-access-6hvxm\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116139    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116155    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116169    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116200    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116215    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116262    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116296    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116325    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-web-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116388    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.116676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.116452    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-config-out\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.217610 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-config-out\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.217610 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217639    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217801    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217859    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217888    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hvxm\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-kube-api-access-6hvxm\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217945    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.217995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218019    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218064    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218230    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218256    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-web-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218507    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.218908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.218570    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.219314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.219124    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.221772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.219798    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.221772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.220714    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86435dba-2559-4b14-963d-df5147012437-config-out\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.221772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.220944    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.221986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.221881    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.221986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.221934    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.222443 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.222280    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.222567 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.222543    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.222935 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.222875    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.222935 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.222881    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.223546 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.223500    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86435dba-2559-4b14-963d-df5147012437-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.223546 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.223523    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.223702 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.223562    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.223702 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.223642    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-web-config\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.224611 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.224586    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.224927 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.224909    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86435dba-2559-4b14-963d-df5147012437-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.227988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.227968    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hvxm\" (UniqueName: \"kubernetes.io/projected/86435dba-2559-4b14-963d-df5147012437-kube-api-access-6hvxm\") pod \"prometheus-k8s-0\" (UID: \"86435dba-2559-4b14-963d-df5147012437\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.380938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.380896    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:14:45.521572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:45.521534    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:14:45.524694 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:14:45.524666    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86435dba_2559_4b14_963d_df5147012437.slice/crio-e915daa050cb8ef770823600562b4381d6958d190c54e2ea100f9fb9d599cee0 WatchSource:0}: Error finding container e915daa050cb8ef770823600562b4381d6958d190c54e2ea100f9fb9d599cee0: Status 404 returned error can't find the container with id e915daa050cb8ef770823600562b4381d6958d190c54e2ea100f9fb9d599cee0
Jul 03 14:14:46.008134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:46.008104    2561 generic.go:356] "Generic (PLEG): container finished" podID="86435dba-2559-4b14-963d-df5147012437" containerID="2c0831bc113325472279be9f7ec641942206c92bb945fdff4b27d8d5157d2ab6" exitCode=0
Jul 03 14:14:46.008306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:46.008143    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerDied","Data":"2c0831bc113325472279be9f7ec641942206c92bb945fdff4b27d8d5157d2ab6"}
Jul 03 14:14:46.008306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:46.008204    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"e915daa050cb8ef770823600562b4381d6958d190c54e2ea100f9fb9d599cee0"}
Jul 03 14:14:46.194165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:46.194126    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de392140-486c-4d79-beea-618ed0b840e1" path="/var/lib/kubelet/pods/de392140-486c-4d79-beea-618ed0b840e1/volumes"
Jul 03 14:14:47.015652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015616    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"cd7499e2fa4105f1798431965a1355fa7fcd2a816d6d02b1a8f1dd3091ad3199"}
Jul 03 14:14:47.016086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015702    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"556dbebdfe76d929879caa7044bd3ae64ab528bc8ee32987542f7cf12e17261b"}
Jul 03 14:14:47.016086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015728    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"5d75daafeb66f4a825a93db5cd99debd5bfd81511fbacd66395a069ef196733a"}
Jul 03 14:14:47.016086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015752    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"a0305ad4107e3cdf88e73dd7a529e6d667ab6bd288d06d9ce3a7dc633c5e292d"}
Jul 03 14:14:47.016086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015783    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"5f167e7ee82eea88284b4f9886f2fd3b601c19fd51e99357aab311f0e7275f1e"}
Jul 03 14:14:47.016086 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.015807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86435dba-2559-4b14-963d-df5147012437","Type":"ContainerStarted","Data":"2cd3bfdf4af2aa4be765cb6dbec620e888442092e44264a20d320cc4bf9e8395"}
Jul 03 14:14:47.048261 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:47.048204    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.029305872 podStartE2EDuration="2.048158253s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.852381ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:14:47.046361964 +0000 UTC m=+189.460258415" watchObservedRunningTime="2026-07-03 14:14:47.048158253 +0000 UTC m=+189.462054706"
Jul 03 14:14:50.381283 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:14:50.381239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:15:45.382012 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:15:45.381952    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:15:45.397784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:15:45.397759    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:15:46.250165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:15:46.250140    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:16:38.026637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:16:38.026604    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:16:38.028288 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:16:38.028269    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:16:38.035985 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:16:38.035959    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 03 14:17:29.123471 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.123393    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"]
Jul 03 14:17:29.126552 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.126537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.129233 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.129212    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 03 14:17:29.130029 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.130013    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-6hmxt\""
Jul 03 14:17:29.130115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.130032    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 03 14:17:29.136102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.136082    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"]
Jul 03 14:17:29.226386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.226357    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9hzv\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-kube-api-access-w9hzv\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.226508 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.226402    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.327703 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.327672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9hzv\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-kube-api-access-w9hzv\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.327871 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.327721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.337523 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.337494    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.337625 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.337593    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9hzv\" (UniqueName: \"kubernetes.io/projected/d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c-kube-api-access-w9hzv\") pod \"cert-manager-cainjector-559cdd8c87-zchl9\" (UID: \"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.445745 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.445675    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"
Jul 03 14:17:29.574043 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.574018    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zchl9"]
Jul 03 14:17:29.577166 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:17:29.577140    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd56ec725_ddb2_4d2f_83fa_0fbb5cd0814c.slice/crio-563c11404894cd6302c577be3c15a21f35c23cc7f76e41525a57c66bc8f33c80 WatchSource:0}: Error finding container 563c11404894cd6302c577be3c15a21f35c23cc7f76e41525a57c66bc8f33c80: Status 404 returned error can't find the container with id 563c11404894cd6302c577be3c15a21f35c23cc7f76e41525a57c66bc8f33c80
Jul 03 14:17:29.579263 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.579243    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:17:29.605018 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:29.604989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9" event={"ID":"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c","Type":"ContainerStarted","Data":"563c11404894cd6302c577be3c15a21f35c23cc7f76e41525a57c66bc8f33c80"}
Jul 03 14:17:33.623467 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:33.623435    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9" event={"ID":"d56ec725-ddb2-4d2f-83fa-0fbb5cd0814c","Type":"ContainerStarted","Data":"181080d993007f3eb04a307a674142851643e6723cd51786fcfc90523e0e8d1e"}
Jul 03 14:17:33.641529 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:33.641484    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zchl9" podStartSLOduration=1.18512108 podStartE2EDuration="4.641469767s" totalImagesPullingTime="3.456348687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:33.639664658 +0000 UTC m=+356.053561107" watchObservedRunningTime="2026-07-03 14:17:33.641469767 +0000 UTC m=+356.055366216"
Jul 03 14:17:39.095003 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.094973    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"]
Jul 03 14:17:39.098175 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.098160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.100790 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.100765    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 03 14:17:39.100958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.100939    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dcfe99-6bbb-442c-a77a-7c2674fb789d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.101033 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.101018    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb7fz\" (UniqueName: \"kubernetes.io/projected/61dcfe99-6bbb-442c-a77a-7c2674fb789d-kube-api-access-tb7fz\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.101732 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.101707    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c6x7h\""
Jul 03 14:17:39.101732 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.101710    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:17:39.107792 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.107774    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"]
Jul 03 14:17:39.202128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.202090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb7fz\" (UniqueName: \"kubernetes.io/projected/61dcfe99-6bbb-442c-a77a-7c2674fb789d-kube-api-access-tb7fz\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.202341 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.202146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dcfe99-6bbb-442c-a77a-7c2674fb789d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.202524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.202505    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61dcfe99-6bbb-442c-a77a-7c2674fb789d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.210833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.210800    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb7fz\" (UniqueName: \"kubernetes.io/projected/61dcfe99-6bbb-442c-a77a-7c2674fb789d-kube-api-access-tb7fz\") pod \"openshift-lws-operator-fd8ccff4c-lfwf8\" (UID: \"61dcfe99-6bbb-442c-a77a-7c2674fb789d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.407539 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.407464    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"
Jul 03 14:17:39.536938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.536915    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8"]
Jul 03 14:17:39.539640 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:17:39.539612    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61dcfe99_6bbb_442c_a77a_7c2674fb789d.slice/crio-0fec06aa5ad93151f2e2c8893fe1c4dfd90b6a0e512781ae22e1ac7007decd7f WatchSource:0}: Error finding container 0fec06aa5ad93151f2e2c8893fe1c4dfd90b6a0e512781ae22e1ac7007decd7f: Status 404 returned error can't find the container with id 0fec06aa5ad93151f2e2c8893fe1c4dfd90b6a0e512781ae22e1ac7007decd7f
Jul 03 14:17:39.647450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:39.647417    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8" event={"ID":"61dcfe99-6bbb-442c-a77a-7c2674fb789d","Type":"ContainerStarted","Data":"0fec06aa5ad93151f2e2c8893fe1c4dfd90b6a0e512781ae22e1ac7007decd7f"}
Jul 03 14:17:42.663055 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:42.663016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8" event={"ID":"61dcfe99-6bbb-442c-a77a-7c2674fb789d","Type":"ContainerStarted","Data":"c376d1166b0030619fac60568c803138dc14dff33045021378f28c4672a78c5a"}
Jul 03 14:17:42.680661 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:17:42.680618    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lfwf8" podStartSLOduration=1.105907481 podStartE2EDuration="3.680602353s" totalImagesPullingTime="2.574694872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:42.679568421 +0000 UTC m=+365.093464872" watchObservedRunningTime="2026-07-03 14:17:42.680602353 +0000 UTC m=+365.094498804"
Jul 03 14:18:03.332770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.332738    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"]
Jul 03 14:18:03.334987 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.334970    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.337371 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.337347    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 03 14:18:03.337575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.337553    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 03 14:18:03.337653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.337577    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 03 14:18:03.337653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.337623    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 03 14:18:03.337765 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.337746    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kf7mt\""
Jul 03 14:18:03.356109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.356073    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"]
Jul 03 14:18:03.385173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.385141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.385315 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.385194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-manifests\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.385315 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.385222    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.385315 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.385291    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-tmp\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.385412 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.385357    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfmrn\" (UniqueName: \"kubernetes.io/projected/077be1f8-be6e-4f15-83a8-4ffb40d2e209-kube-api-access-jfmrn\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.486660 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.486617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-manifests\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.486826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.486670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.486826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.486701    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-tmp\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.486826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.486727    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfmrn\" (UniqueName: \"kubernetes.io/projected/077be1f8-be6e-4f15-83a8-4ffb40d2e209-kube-api-access-jfmrn\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.486826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.486762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.487110 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.487086    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-manifests\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.489238 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.489215    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077be1f8-be6e-4f15-83a8-4ffb40d2e209-tmp\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.489368 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.489352    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.489409 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.489381    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077be1f8-be6e-4f15-83a8-4ffb40d2e209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.497864 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.497836    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfmrn\" (UniqueName: \"kubernetes.io/projected/077be1f8-be6e-4f15-83a8-4ffb40d2e209-kube-api-access-jfmrn\") pod \"opendatahub-operator-controller-manager-6c6554f4cb-7tp42\" (UID: \"077be1f8-be6e-4f15-83a8-4ffb40d2e209\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.644306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.644275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:03.779940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:03.779917    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"]
Jul 03 14:18:03.782422 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:18:03.782396    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077be1f8_be6e_4f15_83a8_4ffb40d2e209.slice/crio-ea3efd9a4badb7dc822d64649ea75446d71ad6911b1447913e2220a9cd649f0e WatchSource:0}: Error finding container ea3efd9a4badb7dc822d64649ea75446d71ad6911b1447913e2220a9cd649f0e: Status 404 returned error can't find the container with id ea3efd9a4badb7dc822d64649ea75446d71ad6911b1447913e2220a9cd649f0e
Jul 03 14:18:04.750619 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:04.750578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42" event={"ID":"077be1f8-be6e-4f15-83a8-4ffb40d2e209","Type":"ContainerStarted","Data":"ea3efd9a4badb7dc822d64649ea75446d71ad6911b1447913e2220a9cd649f0e"}
Jul 03 14:18:06.759833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:06.759793    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42" event={"ID":"077be1f8-be6e-4f15-83a8-4ffb40d2e209","Type":"ContainerStarted","Data":"743997141ecc0c3f7cd74c6f6658ce411a22f19630b0207d64df850d3ab9ab1b"}
Jul 03 14:18:07.765612 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:07.765582    2561 generic.go:356] "Generic (PLEG): container finished" podID="077be1f8-be6e-4f15-83a8-4ffb40d2e209" containerID="743997141ecc0c3f7cd74c6f6658ce411a22f19630b0207d64df850d3ab9ab1b" exitCode=0
Jul 03 14:18:07.765935 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:07.765658    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42" event={"ID":"077be1f8-be6e-4f15-83a8-4ffb40d2e209","Type":"ContainerDied","Data":"743997141ecc0c3f7cd74c6f6658ce411a22f19630b0207d64df850d3ab9ab1b"}
Jul 03 14:18:08.772257 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:08.772219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42" event={"ID":"077be1f8-be6e-4f15-83a8-4ffb40d2e209","Type":"ContainerStarted","Data":"1aef26d8642eacbd28f3f3e8e235f7111c4820287c66efc2a8ec30691eee54a6"}
Jul 03 14:18:08.772846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:08.772326    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:08.796045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:08.795997    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42" podStartSLOduration=1.59061823 podStartE2EDuration="5.795978978s" totalImagesPullingTime="3.130018117s" totalInitContainerRuntime="1.075342631s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:08.794020975 +0000 UTC m=+391.207917425" watchObservedRunningTime="2026-07-03 14:18:08.795978978 +0000 UTC m=+391.209875431"
Jul 03 14:18:19.778780 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:19.778751    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c6554f4cb-7tp42"
Jul 03 14:18:29.887921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.887886    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"]
Jul 03 14:18:29.890969 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.890953    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:29.894339 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.894320    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fdrpt\""
Jul 03 14:18:29.894462 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.894411    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 03 14:18:29.894658 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.894642    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 03 14:18:29.894873 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.894854    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 03 14:18:29.917605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:29.917573    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"]
Jul 03 14:18:30.009413 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009385    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lht2\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-kube-api-access-9lht2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009425    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009533    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009581    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009863 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009625    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009863 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009649    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.009863 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.009701    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110568 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lht2\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-kube-api-access-9lht2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110829    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110916    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.110960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.110945    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.111157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.111123    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.111405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.111379    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.111506 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.111445    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.111572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.111556    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.111768 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.111710    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.113088 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.113070    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.113423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.113405    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.124205 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.124150    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lht2\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-kube-api-access-9lht2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.125372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.125356    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1471b33a-a9f4-4431-af37-fd5ca96ab1df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tv4zj\" (UID: \"1471b33a-a9f4-4431-af37-fd5ca96ab1df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.201344 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.201265    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:30.360336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.360304    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"]
Jul 03 14:18:30.362484 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.362464    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"]
Jul 03 14:18:30.364355 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:18:30.364332    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1471b33a_a9f4_4431_af37_fd5ca96ab1df.slice/crio-fcafdd4539c69314449b2895e94e4d8ce12c324b5e19a6c9b3806b1a3a8f0bb1 WatchSource:0}: Error finding container fcafdd4539c69314449b2895e94e4d8ce12c324b5e19a6c9b3806b1a3a8f0bb1: Status 404 returned error can't find the container with id fcafdd4539c69314449b2895e94e4d8ce12c324b5e19a6c9b3806b1a3a8f0bb1
Jul 03 14:18:30.366954 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.366938    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.375389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.375367    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 03 14:18:30.375720 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.375703    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 03 14:18:30.375817 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.375781    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-j8m2t\""
Jul 03 14:18:30.377647 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.377630    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 03 14:18:30.385820 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.385800    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"]
Jul 03 14:18:30.519795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.519727    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/564f2955-28cb-4b69-a4ef-0e6446705fb8-manager-config\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.519795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.519763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv2vc\" (UniqueName: \"kubernetes.io/projected/564f2955-28cb-4b69-a4ef-0e6446705fb8-kube-api-access-zv2vc\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.519976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.519824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-metrics-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.519976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.519847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.621319 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.621293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv2vc\" (UniqueName: \"kubernetes.io/projected/564f2955-28cb-4b69-a4ef-0e6446705fb8-kube-api-access-zv2vc\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.621495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.621347    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-metrics-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.621495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.621466    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.621618 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.621536    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/564f2955-28cb-4b69-a4ef-0e6446705fb8-manager-config\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.622149 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.622127    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/564f2955-28cb-4b69-a4ef-0e6446705fb8-manager-config\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.623841 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.623812    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-metrics-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.623934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.623871    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/564f2955-28cb-4b69-a4ef-0e6446705fb8-cert\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.630974 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.630952    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv2vc\" (UniqueName: \"kubernetes.io/projected/564f2955-28cb-4b69-a4ef-0e6446705fb8-kube-api-access-zv2vc\") pod \"lws-controller-manager-77cf67f8d7-ggdxv\" (UID: \"564f2955-28cb-4b69-a4ef-0e6446705fb8\") " pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.676869 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.676843    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:30.822269 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.822240    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"]
Jul 03 14:18:30.825062 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:18:30.825038    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564f2955_28cb_4b69_a4ef_0e6446705fb8.slice/crio-b303d863a10af035518acd11b3c9998c23c86c6f529041d3c58a5116f767e174 WatchSource:0}: Error finding container b303d863a10af035518acd11b3c9998c23c86c6f529041d3c58a5116f767e174: Status 404 returned error can't find the container with id b303d863a10af035518acd11b3c9998c23c86c6f529041d3c58a5116f767e174
Jul 03 14:18:30.868809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.868775    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv" event={"ID":"564f2955-28cb-4b69-a4ef-0e6446705fb8","Type":"ContainerStarted","Data":"b303d863a10af035518acd11b3c9998c23c86c6f529041d3c58a5116f767e174"}
Jul 03 14:18:30.869974 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:30.869951    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj" event={"ID":"1471b33a-a9f4-4431-af37-fd5ca96ab1df","Type":"ContainerStarted","Data":"fcafdd4539c69314449b2895e94e4d8ce12c324b5e19a6c9b3806b1a3a8f0bb1"}
Jul 03 14:18:32.505255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.505213    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"]
Jul 03 14:18:32.511041 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.511019    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.514251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.513848    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 03 14:18:32.514251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.514112    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xbjx4\""
Jul 03 14:18:32.523583 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.523521    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"]
Jul 03 14:18:32.537883 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.537846    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkqtl\" (UniqueName: \"kubernetes.io/projected/b1a21974-4c30-4393-8b1f-cbced2867257-kube-api-access-zkqtl\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.538041 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.537908    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a21974-4c30-4393-8b1f-cbced2867257-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.639126 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.639090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkqtl\" (UniqueName: \"kubernetes.io/projected/b1a21974-4c30-4393-8b1f-cbced2867257-kube-api-access-zkqtl\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.639330 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.639131    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a21974-4c30-4393-8b1f-cbced2867257-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.642318 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.642266    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a21974-4c30-4393-8b1f-cbced2867257-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.649070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.649022    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkqtl\" (UniqueName: \"kubernetes.io/projected/b1a21974-4c30-4393-8b1f-cbced2867257-kube-api-access-zkqtl\") pod \"kserve-controller-manager-d6f4b65b8-lgl58\" (UID: \"b1a21974-4c30-4393-8b1f-cbced2867257\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:32.826836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:32.826735    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:33.006469 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:33.006442    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"]
Jul 03 14:18:33.009352 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:18:33.009318    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a21974_4c30_4393_8b1f_cbced2867257.slice/crio-5aa0b5cb98b8a6446eacf3aa0f60b634ca1aea56ac0e9a7dee8b32ad211e8aeb WatchSource:0}: Error finding container 5aa0b5cb98b8a6446eacf3aa0f60b634ca1aea56ac0e9a7dee8b32ad211e8aeb: Status 404 returned error can't find the container with id 5aa0b5cb98b8a6446eacf3aa0f60b634ca1aea56ac0e9a7dee8b32ad211e8aeb
Jul 03 14:18:33.884873 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:33.884830    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58" event={"ID":"b1a21974-4c30-4393-8b1f-cbced2867257","Type":"ContainerStarted","Data":"5aa0b5cb98b8a6446eacf3aa0f60b634ca1aea56ac0e9a7dee8b32ad211e8aeb"}
Jul 03 14:18:34.813765 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:34.813711    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:18:34.813877 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:34.813843    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:18:34.813946 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:34.813887    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:18:34.893365 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:34.893338    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv" event={"ID":"564f2955-28cb-4b69-a4ef-0e6446705fb8","Type":"ContainerStarted","Data":"bd6f85c2aa8cdada890bc1c5f0664b1dc68619bcf137a0c11b45e011f5fd701e"}
Jul 03 14:18:35.898786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:35.898748    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj" event={"ID":"1471b33a-a9f4-4431-af37-fd5ca96ab1df","Type":"ContainerStarted","Data":"443110f4c0ee9bc4f47e98be82493bfd56363dfc76bda927a3104f4311bc59ab"}
Jul 03 14:18:35.921620 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:35.921555    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj" podStartSLOduration=2.474426565 podStartE2EDuration="6.921537958s" totalImagesPullingTime="4.447111393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:35.921372003 +0000 UTC m=+418.335268454" watchObservedRunningTime="2026-07-03 14:18:35.921537958 +0000 UTC m=+418.335434424"
Jul 03 14:18:35.940292 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:35.940227    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv" podStartSLOduration=1.953742166 podStartE2EDuration="5.94021209s" totalImagesPullingTime="3.986469924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:35.939955201 +0000 UTC m=+418.353851652" watchObservedRunningTime="2026-07-03 14:18:35.94021209 +0000 UTC m=+418.354108540"
Jul 03 14:18:36.201950 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:36.201874    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:36.207329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:36.207307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:36.902662 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:36.902643    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:36.903629 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:36.903612    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tv4zj"
Jul 03 14:18:37.908248 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:37.908214    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58" event={"ID":"b1a21974-4c30-4393-8b1f-cbced2867257","Type":"ContainerStarted","Data":"289d78347645b341b5da24fbb8c5ab7e6fb3e3b759d8a2ee9864462340b15722"}
Jul 03 14:18:37.931365 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:37.931318    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58" podStartSLOduration=2.044338837 podStartE2EDuration="5.931302906s" totalImagesPullingTime="3.886964069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:37.930566211 +0000 UTC m=+420.344462662" watchObservedRunningTime="2026-07-03 14:18:37.931302906 +0000 UTC m=+420.345199355"
Jul 03 14:18:38.911935 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:38.911906    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:18:45.899140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:45.899102    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:18:45.901112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:18:45.901092    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77cf67f8d7-ggdxv"
Jul 03 14:19:09.921870 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:19:09.921794    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgl58"
Jul 03 14:20:35.768270 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.768236    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"]
Jul 03 14:20:35.771475 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.771456    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:35.774251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.774225    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 03 14:20:35.774377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.774308    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 03 14:20:35.774779 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.774762    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tmnfd\""
Jul 03 14:20:35.783729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.783708    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"]
Jul 03 14:20:35.844895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.844862    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtg9s\" (UniqueName: \"kubernetes.io/projected/c4e4789a-78e2-48d0-b81d-106cd3c7e116-kube-api-access-dtg9s\") pod \"limitador-operator-controller-manager-859d7b78bd-k9csf\" (UID: \"c4e4789a-78e2-48d0-b81d-106cd3c7e116\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:35.945947 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.945909    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtg9s\" (UniqueName: \"kubernetes.io/projected/c4e4789a-78e2-48d0-b81d-106cd3c7e116-kube-api-access-dtg9s\") pod \"limitador-operator-controller-manager-859d7b78bd-k9csf\" (UID: \"c4e4789a-78e2-48d0-b81d-106cd3c7e116\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:35.956041 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:35.956009    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtg9s\" (UniqueName: \"kubernetes.io/projected/c4e4789a-78e2-48d0-b81d-106cd3c7e116-kube-api-access-dtg9s\") pod \"limitador-operator-controller-manager-859d7b78bd-k9csf\" (UID: \"c4e4789a-78e2-48d0-b81d-106cd3c7e116\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:36.082389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:36.082311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:36.239010 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:36.238980    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"]
Jul 03 14:20:36.242415 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:20:36.242385    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e4789a_78e2_48d0_b81d_106cd3c7e116.slice/crio-d7b84fb1b1d5e8077fdce1ffa8ae5802a1067e6652274935190da026ffe08edf WatchSource:0}: Error finding container d7b84fb1b1d5e8077fdce1ffa8ae5802a1067e6652274935190da026ffe08edf: Status 404 returned error can't find the container with id d7b84fb1b1d5e8077fdce1ffa8ae5802a1067e6652274935190da026ffe08edf
Jul 03 14:20:36.406013 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:36.405984    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf" event={"ID":"c4e4789a-78e2-48d0-b81d-106cd3c7e116","Type":"ContainerStarted","Data":"d7b84fb1b1d5e8077fdce1ffa8ae5802a1067e6652274935190da026ffe08edf"}
Jul 03 14:20:38.417840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:38.417797    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf" event={"ID":"c4e4789a-78e2-48d0-b81d-106cd3c7e116","Type":"ContainerStarted","Data":"8cae23ddc637a9b45b80b28ec21d3898975924fc2a035daa20f30b997207a84f"}
Jul 03 14:20:38.418276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:38.417935    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:20:38.440563 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:38.440517    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf" podStartSLOduration=1.6157129700000001 podStartE2EDuration="3.440502499s" totalImagesPullingTime="1.824789529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:20:38.43868443 +0000 UTC m=+540.852580881" watchObservedRunningTime="2026-07-03 14:20:38.440502499 +0000 UTC m=+540.854398948"
Jul 03 14:20:49.427889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:20:49.427861    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k9csf"
Jul 03 14:21:27.463597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.463565    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:21:27.466698 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.466682    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.469076 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.469040    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 03 14:21:27.469236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.469119    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gxvzb\""
Jul 03 14:21:27.477479 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.477456    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:21:27.481169 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.481144    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqcj\" (UniqueName: \"kubernetes.io/projected/71412083-3365-4fce-9da6-599c6c11879f-kube-api-access-8qqcj\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.481280 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.481202    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71412083-3365-4fce-9da6-599c6c11879f-config-file\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.493665 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.493646    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:21:27.582229 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.582198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqcj\" (UniqueName: \"kubernetes.io/projected/71412083-3365-4fce-9da6-599c6c11879f-kube-api-access-8qqcj\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.582353 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.582237    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71412083-3365-4fce-9da6-599c6c11879f-config-file\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.582829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.582812    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71412083-3365-4fce-9da6-599c6c11879f-config-file\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.591313 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.591280    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqcj\" (UniqueName: \"kubernetes.io/projected/71412083-3365-4fce-9da6-599c6c11879f-kube-api-access-8qqcj\") pod \"limitador-limitador-54d6d988d5-pjbw7\" (UID: \"71412083-3365-4fce-9da6-599c6c11879f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.777173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.777101    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:27.910460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:27.910432    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:21:27.913945 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:21:27.913918    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71412083_3365_4fce_9da6_599c6c11879f.slice/crio-6f1a8f908d9758642e0405323a72b76c85e36e49e13d9c789268522653730558 WatchSource:0}: Error finding container 6f1a8f908d9758642e0405323a72b76c85e36e49e13d9c789268522653730558: Status 404 returned error can't find the container with id 6f1a8f908d9758642e0405323a72b76c85e36e49e13d9c789268522653730558
Jul 03 14:21:28.635047 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:28.634981    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7" event={"ID":"71412083-3365-4fce-9da6-599c6c11879f","Type":"ContainerStarted","Data":"6f1a8f908d9758642e0405323a72b76c85e36e49e13d9c789268522653730558"}
Jul 03 14:21:30.645877 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:30.645836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7" event={"ID":"71412083-3365-4fce-9da6-599c6c11879f","Type":"ContainerStarted","Data":"feb2993997177f3a23ee8ed5a7d667fac901d3ac40b2d97522c1f0a72d14443d"}
Jul 03 14:21:30.646251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:30.645981    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:30.669860 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:30.669783    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7" podStartSLOduration=1.184102888 podStartE2EDuration="3.669769617s" totalImagesPullingTime="2.485666729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:21:30.668513456 +0000 UTC m=+593.082409907" watchObservedRunningTime="2026-07-03 14:21:30.669769617 +0000 UTC m=+593.083666067"
Jul 03 14:21:38.055718 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:38.055687    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:21:38.056132 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:38.056112    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:21:41.651365 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:41.651339    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"
Jul 03 14:21:59.427162 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.427087    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:21:59.432022 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.432005    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.441588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.441571    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zw7mb\""
Jul 03 14:21:59.442406 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.442387    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 03 14:21:59.452284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.452251    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:21:59.555158 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.555129    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.555316 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.555167    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk27c\" (UniqueName: \"kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.656135 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.656103    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.656277 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.656149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk27c\" (UniqueName: \"kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.658754 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.658726    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.666903 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.666871    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk27c\" (UniqueName: \"kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c\") pod \"authorino-76b4467bf5-xcq86\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") " pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.740320 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.740263    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:21:59.869708 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:21:59.869680    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:21:59.873951 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:21:59.873922    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a1ea7d2_9aa3_4a18_b587_1c52dce7e818.slice/crio-0ece3f7e4223d64b3bab04bfc680b4168995faeeee294d166ebb996a3dea9f31 WatchSource:0}: Error finding container 0ece3f7e4223d64b3bab04bfc680b4168995faeeee294d166ebb996a3dea9f31: Status 404 returned error can't find the container with id 0ece3f7e4223d64b3bab04bfc680b4168995faeeee294d166ebb996a3dea9f31
Jul 03 14:22:00.793107 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:22:00.793070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b4467bf5-xcq86" event={"ID":"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818","Type":"ContainerStarted","Data":"0ece3f7e4223d64b3bab04bfc680b4168995faeeee294d166ebb996a3dea9f31"}
Jul 03 14:22:03.816584 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:22:03.816545    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b4467bf5-xcq86" event={"ID":"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818","Type":"ContainerStarted","Data":"fd4963b27d972e51916a58ea03630d39fef26917ffff5c06d09fe75df63f99bf"}
Jul 03 14:22:03.837219 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:22:03.837158    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76b4467bf5-xcq86" podStartSLOduration=1.59536861 podStartE2EDuration="4.837144076s" totalImagesPullingTime="3.241775466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:22:03.834608353 +0000 UTC m=+626.248504802" watchObservedRunningTime="2026-07-03 14:22:03.837144076 +0000 UTC m=+626.251040525"
Jul 03 14:22:46.135338 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:22:46.135307    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:23:34.475704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:23:34.475663    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:23:39.120850 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:23:39.120816    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:23:51.518070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:23:51.518027    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:24:03.031470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.031437    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"]
Jul 03 14:24:03.034749 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.034733    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.038774 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.038753    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 03 14:24:03.039726 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.039699    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-smmbm\""
Jul 03 14:24:03.039726 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.039703    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 03 14:24:03.039879 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.039794    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 03 14:24:03.053759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.053727    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"]
Jul 03 14:24:03.105436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105401    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.105608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.105608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.105608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105521    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/324d87c6-a20e-428a-a590-4ff014cd18ac-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.105608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.105608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.105592    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spv5s\" (UniqueName: \"kubernetes.io/projected/324d87c6-a20e-428a-a590-4ff014cd18ac-kube-api-access-spv5s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206448 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/324d87c6-a20e-428a-a590-4ff014cd18ac-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206594 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206468    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206594 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206526    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spv5s\" (UniqueName: \"kubernetes.io/projected/324d87c6-a20e-428a-a590-4ff014cd18ac-kube-api-access-spv5s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206635    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.206685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.206662    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.207038 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.207001    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.207165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.207061    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.207165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.207128    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.208911 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.208891    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/324d87c6-a20e-428a-a590-4ff014cd18ac-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.209143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.209129    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/324d87c6-a20e-428a-a590-4ff014cd18ac-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.217159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.217140    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spv5s\" (UniqueName: \"kubernetes.io/projected/324d87c6-a20e-428a-a590-4ff014cd18ac-kube-api-access-spv5s\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d\" (UID: \"324d87c6-a20e-428a-a590-4ff014cd18ac\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.344930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.344906    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:03.479403 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.479374    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"]
Jul 03 14:24:03.482247 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:24:03.482215    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324d87c6_a20e_428a_a590_4ff014cd18ac.slice/crio-fa7b2eaf886af4e5fe3581fb4b013e721e0ebed7b90d74223d51189a25b47860 WatchSource:0}: Error finding container fa7b2eaf886af4e5fe3581fb4b013e721e0ebed7b90d74223d51189a25b47860: Status 404 returned error can't find the container with id fa7b2eaf886af4e5fe3581fb4b013e721e0ebed7b90d74223d51189a25b47860
Jul 03 14:24:03.483884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:03.483868    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:24:04.354375 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:04.354333    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d" event={"ID":"324d87c6-a20e-428a-a590-4ff014cd18ac","Type":"ContainerStarted","Data":"fa7b2eaf886af4e5fe3581fb4b013e721e0ebed7b90d74223d51189a25b47860"}
Jul 03 14:24:04.635108 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:04.635034    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:24:05.360417 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:05.360368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d" event={"ID":"324d87c6-a20e-428a-a590-4ff014cd18ac","Type":"ContainerStarted","Data":"7e469b74b5808bb1af3faac7a6fbfafcc9160448d72e6ac2d45c08e542f260a9"}
Jul 03 14:24:05.389714 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:05.389658    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d" podStartSLOduration=1.371560265 podStartE2EDuration="2.389636546s" totalImagesPullingTime="1.018076281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:24:05.387584336 +0000 UTC m=+747.801480787" watchObservedRunningTime="2026-07-03 14:24:05.389636546 +0000 UTC m=+747.803533002"
Jul 03 14:24:11.710312 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:11.710272    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:24:13.345533 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:13.345502    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:13.345931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:13.345558    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:13.358311 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:13.358288    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:13.408896 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:13.408864    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d"
Jul 03 14:24:16.236889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.236853    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"]
Jul 03 14:24:16.242267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.242235    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.247102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.247079    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 03 14:24:16.256752 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.256726    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"]
Jul 03 14:24:16.330507 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.330682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330514    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksqmw\" (UniqueName: \"kubernetes.io/projected/a3e0df26-3f94-48b9-a5b4-38195e276ce4-kube-api-access-ksqmw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.330682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330560    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.330682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330607    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.330682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330632    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.330863 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.330711    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.431838 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.431809    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432014 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.431858    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432014 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.431971    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksqmw\" (UniqueName: \"kubernetes.io/projected/a3e0df26-3f94-48b9-a5b4-38195e276ce4-kube-api-access-ksqmw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432120 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432089    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432328 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432309    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432459 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432427    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.432564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.432463    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.434309 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.434292    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e0df26-3f94-48b9-a5b4-38195e276ce4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.434746 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.434725    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e0df26-3f94-48b9-a5b4-38195e276ce4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.442207 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.442168    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksqmw\" (UniqueName: \"kubernetes.io/projected/a3e0df26-3f94-48b9-a5b4-38195e276ce4-kube-api-access-ksqmw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h\" (UID: \"a3e0df26-3f94-48b9-a5b4-38195e276ce4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.552875 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.552800    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:16.693358 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:16.693328    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"]
Jul 03 14:24:16.697426 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:24:16.697369    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3e0df26_3f94_48b9_a5b4_38195e276ce4.slice/crio-c8edaf0a59836bcfeffd52975c21a805c54502dcce7edcc92f3f48403fd0df68 WatchSource:0}: Error finding container c8edaf0a59836bcfeffd52975c21a805c54502dcce7edcc92f3f48403fd0df68: Status 404 returned error can't find the container with id c8edaf0a59836bcfeffd52975c21a805c54502dcce7edcc92f3f48403fd0df68
Jul 03 14:24:17.417642 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:17.417604    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h" event={"ID":"a3e0df26-3f94-48b9-a5b4-38195e276ce4","Type":"ContainerStarted","Data":"dee03ca48bdfeda8c606bd8411dbc1aa6bf854cc8189be91dd50f09d19159665"}
Jul 03 14:24:17.418013 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:17.417656    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h" event={"ID":"a3e0df26-3f94-48b9-a5b4-38195e276ce4","Type":"ContainerStarted","Data":"c8edaf0a59836bcfeffd52975c21a805c54502dcce7edcc92f3f48403fd0df68"}
Jul 03 14:24:17.436616 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:17.436436    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h" podStartSLOduration=1.207401704 podStartE2EDuration="1.436417909s" totalImagesPullingTime="229.016205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:24:17.435912129 +0000 UTC m=+759.849808578" watchObservedRunningTime="2026-07-03 14:24:17.436417909 +0000 UTC m=+759.850314361"
Jul 03 14:24:18.293931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:18.293895    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:24:26.553633 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:26.553598    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:26.554069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:26.553649    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:26.566396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:26.566374    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:24:27.473387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:24:27.473353    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h"
Jul 03 14:25:37.604542 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.604501    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:25:37.605379 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.605343    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-76b4467bf5-xcq86" podUID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" containerName="authorino" containerID="cri-o://fd4963b27d972e51916a58ea03630d39fef26917ffff5c06d09fe75df63f99bf" gracePeriod=30
Jul 03 14:25:37.775044 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.775009    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" containerID="fd4963b27d972e51916a58ea03630d39fef26917ffff5c06d09fe75df63f99bf" exitCode=0
Jul 03 14:25:37.775165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.775049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b4467bf5-xcq86" event={"ID":"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818","Type":"ContainerDied","Data":"fd4963b27d972e51916a58ea03630d39fef26917ffff5c06d09fe75df63f99bf"}
Jul 03 14:25:37.856399 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.856348    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:25:37.923111 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.923084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk27c\" (UniqueName: \"kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c\") pod \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") "
Jul 03 14:25:37.923276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.923122    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert\") pod \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\" (UID: \"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818\") "
Jul 03 14:25:37.925372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.925336    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c" (OuterVolumeSpecName: "kube-api-access-vk27c") pod "3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" (UID: "3a1ea7d2-9aa3-4a18-b587-1c52dce7e818"). InnerVolumeSpecName "kube-api-access-vk27c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:25:37.933887 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:37.933864    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" (UID: "3a1ea7d2-9aa3-4a18-b587-1c52dce7e818"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:25:38.024218 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.024166    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk27c\" (UniqueName: \"kubernetes.io/projected/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-kube-api-access-vk27c\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:25:38.024218 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.024210    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818-tls-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:25:38.780951 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.780922    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b4467bf5-xcq86" event={"ID":"3a1ea7d2-9aa3-4a18-b587-1c52dce7e818","Type":"ContainerDied","Data":"0ece3f7e4223d64b3bab04bfc680b4168995faeeee294d166ebb996a3dea9f31"}
Jul 03 14:25:38.780951 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.780947    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b4467bf5-xcq86"
Jul 03 14:25:38.781439 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.780963    2561 scope.go:117] "RemoveContainer" containerID="fd4963b27d972e51916a58ea03630d39fef26917ffff5c06d09fe75df63f99bf"
Jul 03 14:25:38.801916 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.801888    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:25:38.809076 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:38.809051    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76b4467bf5-xcq86"]
Jul 03 14:25:40.190555 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:25:40.190524    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" path="/var/lib/kubelet/pods/3a1ea7d2-9aa3-4a18-b587-1c52dce7e818/volumes"
Jul 03 14:26:24.402170 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:24.402134    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:26:35.605025 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:35.604991    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:26:38.081385 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:38.081358    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:26:38.083977 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:38.083958    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:26:46.104435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:46.104406    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:26:54.501431 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:26:54.501396    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:27:06.093232 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:27:06.093197    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:28:13.903959 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:28:13.903883    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:28:23.510333 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:28:23.510298    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:28:32.105923 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:28:32.105884    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:28:40.801588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:28:40.801551    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:28:51.008064 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:28:51.008028    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:29:01.207606 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:29:01.207576    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:29:12.721806 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:29:12.721768    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:29:22.298772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:29:22.298736    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:29:44.907288 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:29:44.907255    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:29:54.294951 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:29:54.294916    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:30:00.222221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.222170    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-7rwdn"]
Jul 03 14:30:00.222598 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.222520    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" containerName="authorino"
Jul 03 14:30:00.222598 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.222533    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" containerName="authorino"
Jul 03 14:30:00.222668 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.222601    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a1ea7d2-9aa3-4a18-b587-1c52dce7e818" containerName="authorino"
Jul 03 14:30:00.225643 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.225624    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:00.231204 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.231158    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:30:00.343634 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.343601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-7rwdn"]
Jul 03 14:30:00.390273 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.390241    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpt9h\" (UniqueName: \"kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h\") pod \"maas-api-key-cleanup-29718150-7rwdn\" (UID: \"15731c11-6060-49a8-8b6f-e89c5f0a3b47\") " pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:00.491118 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.491030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpt9h\" (UniqueName: \"kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h\") pod \"maas-api-key-cleanup-29718150-7rwdn\" (UID: \"15731c11-6060-49a8-8b6f-e89c5f0a3b47\") " pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:00.508325 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.508294    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpt9h\" (UniqueName: \"kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h\") pod \"maas-api-key-cleanup-29718150-7rwdn\" (UID: \"15731c11-6060-49a8-8b6f-e89c5f0a3b47\") " pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:00.535584 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.535562    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:00.682962 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.682932    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-7rwdn"]
Jul 03 14:30:00.685991 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:30:00.685966    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15731c11_6060_49a8_8b6f_e89c5f0a3b47.slice/crio-aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361 WatchSource:0}: Error finding container aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361: Status 404 returned error can't find the container with id aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361
Jul 03 14:30:00.687943 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.687924    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:30:00.966556 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:00.966520    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn" event={"ID":"15731c11-6060-49a8-8b6f-e89c5f0a3b47","Type":"ContainerStarted","Data":"aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361"}
Jul 03 14:30:03.984332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:03.984295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn" event={"ID":"15731c11-6060-49a8-8b6f-e89c5f0a3b47","Type":"ContainerStarted","Data":"b3548e55e73ddc85a58dbf168ef2ced106bc4ef7ce5ef850f087e42efebebf60"}
Jul 03 14:30:04.003863 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:04.003813    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn" podStartSLOduration=1.116029304 podStartE2EDuration="4.003797758s" totalImagesPullingTime="2.887768454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:30:04.001730248 +0000 UTC m=+1106.415626686" watchObservedRunningTime="2026-07-03 14:30:04.003797758 +0000 UTC m=+1106.417694207"
Jul 03 14:30:04.990435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:04.990398    2561 generic.go:356] "Generic (PLEG): container finished" podID="15731c11-6060-49a8-8b6f-e89c5f0a3b47" containerID="b3548e55e73ddc85a58dbf168ef2ced106bc4ef7ce5ef850f087e42efebebf60" exitCode=0
Jul 03 14:30:04.990809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:04.990486    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn" event={"ID":"15731c11-6060-49a8-8b6f-e89c5f0a3b47","Type":"ContainerDied","Data":"b3548e55e73ddc85a58dbf168ef2ced106bc4ef7ce5ef850f087e42efebebf60"}
Jul 03 14:30:06.121748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:06.121721    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:06.245455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:06.245366    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpt9h\" (UniqueName: \"kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h\") pod \"15731c11-6060-49a8-8b6f-e89c5f0a3b47\" (UID: \"15731c11-6060-49a8-8b6f-e89c5f0a3b47\") "
Jul 03 14:30:06.247592 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:06.247568    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h" (OuterVolumeSpecName: "kube-api-access-qpt9h") pod "15731c11-6060-49a8-8b6f-e89c5f0a3b47" (UID: "15731c11-6060-49a8-8b6f-e89c5f0a3b47"). InnerVolumeSpecName "kube-api-access-qpt9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:30:06.346792 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:06.346758    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpt9h\" (UniqueName: \"kubernetes.io/projected/15731c11-6060-49a8-8b6f-e89c5f0a3b47-kube-api-access-qpt9h\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:30:07.000519 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:07.000481    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn"
Jul 03 14:30:07.000688 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:07.000533    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-7rwdn" event={"ID":"15731c11-6060-49a8-8b6f-e89c5f0a3b47","Type":"ContainerDied","Data":"aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361"}
Jul 03 14:30:07.000688 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:07.000579    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aed5d14c6e1f8c58743d8a5ceb111d1800ca3350eb31f764c73ee211f0141361"
Jul 03 14:30:22.428660 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:22.428631    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:30:32.908172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:32.908138    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:30:47.719518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:47.719483    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:30:59.599704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:30:59.599626    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:31:09.209097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:31:09.209059    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:31:17.798753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:31:17.798711    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:31:26.899297 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:31:26.899261    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:31:38.108045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:31:38.108021    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:31:38.111973 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:31:38.111946    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:33:14.205276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:33:14.205237    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:33:23.403889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:33:23.403854    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:33:35.805492 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:33:35.805457    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:33:46.998884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:33:46.998847    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:33:57.406250 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:33:57.406213    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:07.523382 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:07.523348    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:15.412882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:15.412851    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:24.499743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:24.499701    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:32.911747 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:32.911713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:43.697783 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:43.697738    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:34:50.707217 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:34:50.707168    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:35:02.094618 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:35:02.094582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:35:23.409659 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:35:23.409626    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:35:32.106090 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:35:32.106050    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:35:42.501426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:35:42.501391    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:35:54.607610 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:35:54.607578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:09.822000 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:09.821970    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:20.409443 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:20.409414    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:29.694768 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:29.694730    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:38.143508 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:38.143469    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:36:38.148172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:38.148146    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:36:40.002755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:40.002714    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:48.803888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:48.803852    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:36:58.108032 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:36:58.107994    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:37:12.100666 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:37:12.100627    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:37:20.807627 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:37:20.807592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:37:30.003509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:37:30.003478    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:37:41.910026 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:37:41.909981    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:01.594244 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:01.594205    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:13.401908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:13.401870    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:26.496140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:26.496103    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:39.001934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:39.001898    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:50.920586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.920555    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:38:50.921084 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.921067    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15731c11-6060-49a8-8b6f-e89c5f0a3b47" containerName="cleanup"
Jul 03 14:38:50.921161 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.921086    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="15731c11-6060-49a8-8b6f-e89c5f0a3b47" containerName="cleanup"
Jul 03 14:38:50.921161 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.921150    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="15731c11-6060-49a8-8b6f-e89c5f0a3b47" containerName="cleanup"
Jul 03 14:38:50.924018 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.923999    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:50.927309 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.927292    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-69718436-openshift-default-dockercfg-wknqq\""
Jul 03 14:38:50.947174 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:50.947149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:38:51.021943 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.021909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.021954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.021972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022023    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24f49\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022353 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022140    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022353 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022165    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.022353 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.022207    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123263 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123230    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123271    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123294    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123313    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123337    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24f49\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123790    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123886 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123825    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.123969 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.123949    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.124286 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.124259    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.124389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.124314    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.125829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.125803    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.126098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.126080    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.136056 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.136029    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.136143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.136056    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24f49\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49\") pod \"e2e-ait-69718436-openshift-default-8c7d45767-btv6k\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") " pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.235732 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.235663    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:51.373590 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.373551    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:38:51.376152 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:38:51.376124    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7821da67_a485_492f_882e_cb8fe8d4d116.slice/crio-89d8252f1cfd3e7655f443f56f040583e161f08f077d6fbbf0d6436db4127738 WatchSource:0}: Error finding container 89d8252f1cfd3e7655f443f56f040583e161f08f077d6fbbf0d6436db4127738: Status 404 returned error can't find the container with id 89d8252f1cfd3e7655f443f56f040583e161f08f077d6fbbf0d6436db4127738
Jul 03 14:38:51.378114 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.378083    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:38:51.378211 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.378150    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:38:51.378211 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:51.378198    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:38:52.376553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:52.376517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" event={"ID":"7821da67-a485-492f-882e-cb8fe8d4d116","Type":"ContainerStarted","Data":"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"}
Jul 03 14:38:52.376914 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:52.376570    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" event={"ID":"7821da67-a485-492f-882e-cb8fe8d4d116","Type":"ContainerStarted","Data":"89d8252f1cfd3e7655f443f56f040583e161f08f077d6fbbf0d6436db4127738"}
Jul 03 14:38:52.400496 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:52.400448    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" podStartSLOduration=2.400433791 podStartE2EDuration="2.400433791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:38:52.398306151 +0000 UTC m=+1634.812202604" watchObservedRunningTime="2026-07-03 14:38:52.400433791 +0000 UTC m=+1634.814330240"
Jul 03 14:38:53.236261 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:53.236224    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:53.241618 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:53.241593    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:53.382065 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:53.382033    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:53.382999 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:53.382979    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:38:54.898915 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:54.898879    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:38:57.525329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:57.525289    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:38:57.525779 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:38:57.525479    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" podUID="7821da67-a485-492f-882e-cb8fe8d4d116" containerName="istio-proxy" containerID="cri-o://6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4" gracePeriod=30
Jul 03 14:39:02.869732 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.869706    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:39:02.935389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935309    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935346    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935385    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935636 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935501    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935636 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935553    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935636 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935585    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935636 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935612    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24f49\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935668    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935702    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy\") pod \"7821da67-a485-492f-882e-cb8fe8d4d116\" (UID: \"7821da67-a485-492f-882e-cb8fe8d4d116\") "
Jul 03 14:39:02.935826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935782    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:39:02.935978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.935951    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:02.936058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.936023    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7821da67-a485-492f-882e-cb8fe8d4d116-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:02.936058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.936041    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:02.936195 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.936160    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:02.936470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.936443    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data" (OuterVolumeSpecName: "istio-data") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:02.936470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.936459    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:02.937890 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.937863    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token" (OuterVolumeSpecName: "istio-token") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:02.937978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.937941    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49" (OuterVolumeSpecName: "kube-api-access-24f49") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "kube-api-access-24f49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:02.937978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.937944    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:39:02.938142 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:02.938125    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7821da67-a485-492f-882e-cb8fe8d4d116" (UID: "7821da67-a485-492f-882e-cb8fe8d4d116"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:03.036872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036827    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.036872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036866    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.036872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036876    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.036872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036884    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7821da67-a485-492f-882e-cb8fe8d4d116-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.037131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036893    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.037131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036900    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7821da67-a485-492f-882e-cb8fe8d4d116-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.037131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.036908    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24f49\" (UniqueName: \"kubernetes.io/projected/7821da67-a485-492f-882e-cb8fe8d4d116-kube-api-access-24f49\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:03.429414 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.429376    2561 generic.go:356] "Generic (PLEG): container finished" podID="7821da67-a485-492f-882e-cb8fe8d4d116" containerID="6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4" exitCode=0
Jul 03 14:39:03.429597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.429424    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"
Jul 03 14:39:03.429597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.429459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" event={"ID":"7821da67-a485-492f-882e-cb8fe8d4d116","Type":"ContainerDied","Data":"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"}
Jul 03 14:39:03.429597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.429519    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k" event={"ID":"7821da67-a485-492f-882e-cb8fe8d4d116","Type":"ContainerDied","Data":"89d8252f1cfd3e7655f443f56f040583e161f08f077d6fbbf0d6436db4127738"}
Jul 03 14:39:03.429597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.429541    2561 scope.go:117] "RemoveContainer" containerID="6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"
Jul 03 14:39:03.440685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.440664    2561 scope.go:117] "RemoveContainer" containerID="6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"
Jul 03 14:39:03.441172 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:03.441129    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4\": container with ID starting with 6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4 not found: ID does not exist" containerID="6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"
Jul 03 14:39:03.441306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.441199    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4"} err="failed to get container status \"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4\": rpc error: code = NotFound desc = could not find container \"6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4\": container with ID starting with 6565f5f3d27bef480c990ccc5af0932436b69a0db483b12f7edbba2d137431f4 not found: ID does not exist"
Jul 03 14:39:03.454310 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.454282    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:39:03.458594 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.458573    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-69718436-openshift-default-8c7d45767-btv6k"]
Jul 03 14:39:03.601421 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:03.601379    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:39:04.066928 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.066899    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:04.067471 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.067444    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7821da67-a485-492f-882e-cb8fe8d4d116" containerName="istio-proxy"
Jul 03 14:39:04.067471 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.067468    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7821da67-a485-492f-882e-cb8fe8d4d116" containerName="istio-proxy"
Jul 03 14:39:04.067619 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.067594    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7821da67-a485-492f-882e-cb8fe8d4d116" containerName="istio-proxy"
Jul 03 14:39:04.070572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.070544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.073430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.073397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2a3940ec-openshift-default-dockercfg-rdnp5\""
Jul 03 14:39:04.090436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.090411    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:04.146982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.146955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.146985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147229 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147137    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvr5m\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147229 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147172    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147244    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.147296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.147281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.191275 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.191248    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7821da67-a485-492f-882e-cb8fe8d4d116" path="/var/lib/kubelet/pods/7821da67-a485-492f-882e-cb8fe8d4d116/volumes"
Jul 03 14:39:04.247795 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.247770    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvr5m\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.247911 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.247812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.247911 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.247844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248022 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.247970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248075 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248135 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248084    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248135 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248113    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248238    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248414 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248325    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248600 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248531    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248600 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248580    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.248867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.248750    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.250403 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.250384    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.250540 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.250524    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.256397 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.256377    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.256998 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.256977    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvr5m\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m\") pod \"e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") " pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.383902 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.383872    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:04.518216 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.518169    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:04.521173 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:39:04.521144    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d97d0d_9f31_4fc4_9b6f_c37bd6f993fc.slice/crio-fbea7223a8acefe16086f67aafe984219dc1ad935ae9cd80aadadee56b5c489d WatchSource:0}: Error finding container fbea7223a8acefe16086f67aafe984219dc1ad935ae9cd80aadadee56b5c489d: Status 404 returned error can't find the container with id fbea7223a8acefe16086f67aafe984219dc1ad935ae9cd80aadadee56b5c489d
Jul 03 14:39:04.523258 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.523225    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:04.523344 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.523293    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:04.523344 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:04.523324    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:05.441952 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.441919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerStarted","Data":"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"}
Jul 03 14:39:05.442335 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.441968    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerStarted","Data":"fbea7223a8acefe16086f67aafe984219dc1ad935ae9cd80aadadee56b5c489d"}
Jul 03 14:39:05.469652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.469591    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" podStartSLOduration=1.469570566 podStartE2EDuration="1.469570566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:39:05.46778663 +0000 UTC m=+1647.881683081" watchObservedRunningTime="2026-07-03 14:39:05.469570566 +0000 UTC m=+1647.883467012"
Jul 03 14:39:05.486871 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.486840    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:05.489113 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.489090    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.491588 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.491564    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:39:05.491709 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.491568    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-2a3940ec-serving-cert\""
Jul 03 14:39:05.500553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.500530    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:05.556562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.556531    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbb7\" (UniqueName: \"kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.556707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.556573    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.556707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.556650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.657791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.657750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbb7\" (UniqueName: \"kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.657992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.657806    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.657992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.657856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.658475 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.658452    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.660621 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.660597    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.668205 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.668169    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbb7\" (UniqueName: \"kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7\") pod \"maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") " pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.800828 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.800741    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:05.939600 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.939574    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:05.942228 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:39:05.942200    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd85c87_59e7_41a3_9736_dc9c8dff4691.slice/crio-ea6ed51eac4fd8a169dfe1f32e913f4648f24aec9fcdbf7c2b3357d9e067acab WatchSource:0}: Error finding container ea6ed51eac4fd8a169dfe1f32e913f4648f24aec9fcdbf7c2b3357d9e067acab: Status 404 returned error can't find the container with id ea6ed51eac4fd8a169dfe1f32e913f4648f24aec9fcdbf7c2b3357d9e067acab
Jul 03 14:39:05.943669 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:05.943649    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:39:06.384053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:06.384018    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:06.388882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:06.388859    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:06.447521 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:06.447481    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" event={"ID":"7bd85c87-59e7-41a3-9736-dc9c8dff4691","Type":"ContainerStarted","Data":"ea6ed51eac4fd8a169dfe1f32e913f4648f24aec9fcdbf7c2b3357d9e067acab"}
Jul 03 14:39:06.447927 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:06.447684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:06.448722 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:06.448700    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:08.462140 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:08.462106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" event={"ID":"7bd85c87-59e7-41a3-9736-dc9c8dff4691","Type":"ContainerStarted","Data":"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"}
Jul 03 14:39:08.462627 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:08.462390    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:08.485265 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:08.485211    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" podStartSLOduration=1.871784509 podStartE2EDuration="3.485175238s" totalImagesPullingTime="1.613390729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:39:08.482898256 +0000 UTC m=+1650.896794733" watchObservedRunningTime="2026-07-03 14:39:08.485175238 +0000 UTC m=+1650.899071689"
Jul 03 14:39:10.864788 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:10.864744    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:10.865445 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:10.864984    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" podUID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" containerName="maas-api" containerID="cri-o://87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9" gracePeriod=30
Jul 03 14:39:11.127444 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.127392    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:11.209938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.209907    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca\") pod \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") "
Jul 03 14:39:11.209938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.209945    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls\") pod \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") "
Jul 03 14:39:11.210208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.210012    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfbb7\" (UniqueName: \"kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7\") pod \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\" (UID: \"7bd85c87-59e7-41a3-9736-dc9c8dff4691\") "
Jul 03 14:39:11.210411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.210377    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bd85c87-59e7-41a3-9736-dc9c8dff4691" (UID: "7bd85c87-59e7-41a3-9736-dc9c8dff4691"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:39:11.212245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.212214    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7" (OuterVolumeSpecName: "kube-api-access-hfbb7") pod "7bd85c87-59e7-41a3-9736-dc9c8dff4691" (UID: "7bd85c87-59e7-41a3-9736-dc9c8dff4691"). InnerVolumeSpecName "kube-api-access-hfbb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:11.212328 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.212243    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bd85c87-59e7-41a3-9736-dc9c8dff4691" (UID: "7bd85c87-59e7-41a3-9736-dc9c8dff4691"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:39:11.311520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.311490    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd85c87-59e7-41a3-9736-dc9c8dff4691-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:11.311520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.311516    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd85c87-59e7-41a3-9736-dc9c8dff4691-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:11.311520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.311526    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfbb7\" (UniqueName: \"kubernetes.io/projected/7bd85c87-59e7-41a3-9736-dc9c8dff4691-kube-api-access-hfbb7\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:11.482317 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.482219    2561 generic.go:356] "Generic (PLEG): container finished" podID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" containerID="87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9" exitCode=0
Jul 03 14:39:11.482317 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.482279    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"
Jul 03 14:39:11.482536 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.482317    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" event={"ID":"7bd85c87-59e7-41a3-9736-dc9c8dff4691","Type":"ContainerDied","Data":"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"}
Jul 03 14:39:11.482536 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.482372    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8" event={"ID":"7bd85c87-59e7-41a3-9736-dc9c8dff4691","Type":"ContainerDied","Data":"ea6ed51eac4fd8a169dfe1f32e913f4648f24aec9fcdbf7c2b3357d9e067acab"}
Jul 03 14:39:11.482536 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.482394    2561 scope.go:117] "RemoveContainer" containerID="87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"
Jul 03 14:39:11.491708 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.491693    2561 scope.go:117] "RemoveContainer" containerID="87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"
Jul 03 14:39:11.491970 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:11.491949    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9\": container with ID starting with 87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9 not found: ID does not exist" containerID="87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"
Jul 03 14:39:11.492062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.491977    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9"} err="failed to get container status \"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9\": rpc error: code = NotFound desc = could not find container \"87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9\": container with ID starting with 87569434f6ea14530d7b90fec30c3d1d946f0da20ec70cc04d99c7adeb2af8c9 not found: ID does not exist"
Jul 03 14:39:11.508198 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.505478    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:11.511035 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:11.511012    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-2a3940ec-74c985d8d9-xqxs8"]
Jul 03 14:39:12.190700 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:12.190660    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" path="/var/lib/kubelet/pods/7bd85c87-59e7-41a3-9736-dc9c8dff4691/volumes"
Jul 03 14:39:16.508738 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.508710    2561 generic.go:356] "Generic (PLEG): container finished" podID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerID="bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c" exitCode=0
Jul 03 14:39:16.509141 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.508785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerDied","Data":"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"}
Jul 03 14:39:16.509141 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.509135    2561 scope.go:117] "RemoveContainer" containerID="bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"
Jul 03 14:39:16.510057 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.510029    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:16.510151 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.510111    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:16.510240 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.510160    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:16.785016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:16.784930    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:17.515049 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:17.515013    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerStarted","Data":"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"}
Jul 03 14:39:17.515525 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:17.515128    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy" containerID="cri-o://fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6" gracePeriod=30
Jul 03 14:39:21.447692 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:21.447650    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:22.759400 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.759377    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:22.816729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816660    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816694    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816718    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816756    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816794    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvr5m\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816816    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816831    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816900    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.816982 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.816920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data\") pod \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\" (UID: \"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc\") "
Jul 03 14:39:22.817303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817104    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:22.817303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817236    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.817408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817343    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:22.817408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817353    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:22.817687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817414    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:39:22.817922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.817902    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:22.819067 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.819047    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:22.819207 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.819139    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:22.819479 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.819453    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m" (OuterVolumeSpecName: "kube-api-access-dvr5m") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "kube-api-access-dvr5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:22.819479 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.819455    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" (UID: "77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:39:22.917892 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917868    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.917892 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917892    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917904    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917918    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917929    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917942    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvr5m\" (UniqueName: \"kubernetes.io/projected/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-kube-api-access-dvr5m\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917955    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:22.918053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:22.917965    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:23.187494 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187468    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:23.187819 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187807    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:23.187872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187822    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:23.187872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187833    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" containerName="maas-api"
Jul 03 14:39:23.187872 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187840    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" containerName="maas-api"
Jul 03 14:39:23.187970 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187913    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:23.187970 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187927    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:23.187970 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.187936    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd85c87-59e7-41a3-9736-dc9c8dff4691" containerName="maas-api"
Jul 03 14:39:23.190006 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.189991    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.193664 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.193647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ea235c7d-openshift-default-dockercfg-8vhbp\""
Jul 03 14:39:23.204869 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.204847    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:23.321767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321735    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321767    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321808    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321857    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321872    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321890    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.321965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.321915    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rps79\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.322149 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.322011    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.422867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.422840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.422867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.422870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.422892    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.422906    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.422961    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423019    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps79\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423225    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423342    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423361    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423405    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.423688 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.423633    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.424065 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.424039    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.425689 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.425636    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.425796 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.425775    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.437238 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.437213    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.437585 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.437564    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rps79\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79\") pod \"e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") " pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.500100 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.500079    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:23.545447 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.545413    2561 generic.go:356] "Generic (PLEG): container finished" podID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerID="fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6" exitCode=0
Jul 03 14:39:23.545623 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.545463    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerDied","Data":"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"}
Jul 03 14:39:23.545623 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.545486    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"
Jul 03 14:39:23.545623 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.545504    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7" event={"ID":"77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc","Type":"ContainerDied","Data":"fbea7223a8acefe16086f67aafe984219dc1ad935ae9cd80aadadee56b5c489d"}
Jul 03 14:39:23.545623 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.545526    2561 scope.go:117] "RemoveContainer" containerID="fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"
Jul 03 14:39:23.558378 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.558356    2561 scope.go:117] "RemoveContainer" containerID="bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"
Jul 03 14:39:23.571329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.571299    2561 scope.go:117] "RemoveContainer" containerID="fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"
Jul 03 14:39:23.571564 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:23.571546    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6\": container with ID starting with fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6 not found: ID does not exist" containerID="fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"
Jul 03 14:39:23.571603 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.571577    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6"} err="failed to get container status \"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6\": rpc error: code = NotFound desc = could not find container \"fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6\": container with ID starting with fc9ab26f7d3f2e1ef32c5c7fa741e9018c8f238ed158e336d6bd40832ba5edb6 not found: ID does not exist"
Jul 03 14:39:23.571652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.571602    2561 scope.go:117] "RemoveContainer" containerID="bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"
Jul 03 14:39:23.571873 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:23.571847    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c\": container with ID starting with bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c not found: ID does not exist" containerID="bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"
Jul 03 14:39:23.571934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.571884    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c"} err="failed to get container status \"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c\": rpc error: code = NotFound desc = could not find container \"bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c\": container with ID starting with bd78346587fa58ea0d7c5f6052f228c94505952b2e093ee88017f5142e55e50c not found: ID does not exist"
Jul 03 14:39:23.576978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.576941    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:23.579525 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.579485    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2a3940ec-openshift-default-85d564c57-b4px7"]
Jul 03 14:39:23.646410 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.646307    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:23.648620 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:39:23.648589    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4ec242_31c5_424e_9c0c_7aa638f8b9cc.slice/crio-7a565e0f0f204add082f7b2db520c9f065abef0cc3429e2f2484669f7846c412 WatchSource:0}: Error finding container 7a565e0f0f204add082f7b2db520c9f065abef0cc3429e2f2484669f7846c412: Status 404 returned error can't find the container with id 7a565e0f0f204add082f7b2db520c9f065abef0cc3429e2f2484669f7846c412
Jul 03 14:39:23.651004 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.650976    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:23.651106 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.651051    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:23.651106 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:23.651099    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:39:24.190993 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.190961    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" path="/var/lib/kubelet/pods/77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc/volumes"
Jul 03 14:39:24.552238 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.552146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" event={"ID":"be4ec242-31c5-424e-9c0c-7aa638f8b9cc","Type":"ContainerStarted","Data":"e984d62de12219dec6427c9ec279de1635669c857cb2f1fa731f17187b5d355d"}
Jul 03 14:39:24.552238 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.552205    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" event={"ID":"be4ec242-31c5-424e-9c0c-7aa638f8b9cc","Type":"ContainerStarted","Data":"7a565e0f0f204add082f7b2db520c9f065abef0cc3429e2f2484669f7846c412"}
Jul 03 14:39:24.576017 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.575961    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" podStartSLOduration=1.575945757 podStartE2EDuration="1.575945757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:39:24.573287814 +0000 UTC m=+1666.987184266" watchObservedRunningTime="2026-07-03 14:39:24.575945757 +0000 UTC m=+1666.989842206"
Jul 03 14:39:24.710165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.710118    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:24.710710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.710683    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:24.710710 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.710710    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d97d0d-9f31-4fc4-9b6f-c37bd6f993fc" containerName="istio-proxy"
Jul 03 14:39:24.713263 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.713242    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.716044 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.716024    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-ea235c7d-serving-cert\""
Jul 03 14:39:24.716152 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.716134    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:39:24.723135 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.723114    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:24.835415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.835393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.835575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.835450    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww8kw\" (UniqueName: \"kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.835575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.835542    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.936384 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.936349    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.936569 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.936402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.936569 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.936456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww8kw\" (UniqueName: \"kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.936569 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:24.936556    2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-ea235c7d-serving-cert: secret "maas-api-e2e-derive-ea235c7d-serving-cert" not found
Jul 03 14:39:24.936737 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:24.936626    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls podName:f3cbace6-6f72-4943-ad98-788afd9779c2 nodeName:}" failed. No retries permitted until 2026-07-03 14:39:25.43660979 +0000 UTC m=+1667.850506217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls") pod "maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" (UID: "f3cbace6-6f72-4943-ad98-788afd9779c2") : secret "maas-api-e2e-derive-ea235c7d-serving-cert" not found
Jul 03 14:39:24.937009 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.936990    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:24.959240 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:24.959221    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww8kw\" (UniqueName: \"kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:25.441508 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.441476    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:25.444008 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.443970    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") pod \"maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") " pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:25.500348 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.500320    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:25.504916 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.504892    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:25.557158 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.557126    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:25.557876 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.557857    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:25.627586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.627559    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:25.783555 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:25.783532    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:25.785755 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:39:25.785725    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3cbace6_6f72_4943_ad98_788afd9779c2.slice/crio-0b78c0b09f734d70fe8a4b636c406a2c6b5d34fec7c613ea1819b26d6bded89b WatchSource:0}: Error finding container 0b78c0b09f734d70fe8a4b636c406a2c6b5d34fec7c613ea1819b26d6bded89b: Status 404 returned error can't find the container with id 0b78c0b09f734d70fe8a4b636c406a2c6b5d34fec7c613ea1819b26d6bded89b
Jul 03 14:39:26.564550 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:26.564455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" event={"ID":"f3cbace6-6f72-4943-ad98-788afd9779c2","Type":"ContainerStarted","Data":"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"}
Jul 03 14:39:26.564550 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:26.564504    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" event={"ID":"f3cbace6-6f72-4943-ad98-788afd9779c2","Type":"ContainerStarted","Data":"0b78c0b09f734d70fe8a4b636c406a2c6b5d34fec7c613ea1819b26d6bded89b"}
Jul 03 14:39:26.565015 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:26.564626    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:26.584904 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:26.584862    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" podStartSLOduration=2.202988137 podStartE2EDuration="2.584850129s" totalImagesPullingTime="381.861992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:39:26.582468402 +0000 UTC m=+1668.996364852" watchObservedRunningTime="2026-07-03 14:39:26.584850129 +0000 UTC m=+1668.998746579"
Jul 03 14:39:27.290914 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:27.290884    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:39:28.994270 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:28.994231    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:28.994696 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:28.994442    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" podUID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" containerName="istio-proxy" containerID="cri-o://e984d62de12219dec6427c9ec279de1635669c857cb2f1fa731f17187b5d355d" gracePeriod=30
Jul 03 14:39:29.258274 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.257714    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:29.258274 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.257941    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" podUID="f3cbace6-6f72-4943-ad98-788afd9779c2" containerName="maas-api" containerID="cri-o://66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d" gracePeriod=30
Jul 03 14:39:29.521796 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.521774    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:29.581052 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581026    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww8kw\" (UniqueName: \"kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw\") pod \"f3cbace6-6f72-4943-ad98-788afd9779c2\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") "
Jul 03 14:39:29.581234 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581077    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") pod \"f3cbace6-6f72-4943-ad98-788afd9779c2\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") "
Jul 03 14:39:29.581234 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581156    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca\") pod \"f3cbace6-6f72-4943-ad98-788afd9779c2\" (UID: \"f3cbace6-6f72-4943-ad98-788afd9779c2\") "
Jul 03 14:39:29.581382 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581356    2561 generic.go:356] "Generic (PLEG): container finished" podID="f3cbace6-6f72-4943-ad98-788afd9779c2" containerID="66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d" exitCode=0
Jul 03 14:39:29.581498 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581476    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" event={"ID":"f3cbace6-6f72-4943-ad98-788afd9779c2","Type":"ContainerDied","Data":"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"}
Jul 03 14:39:29.582153 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581486    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"
Jul 03 14:39:29.582153 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581756    2561 scope.go:117] "RemoveContainer" containerID="66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"
Jul 03 14:39:29.582153 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.581661    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3cbace6-6f72-4943-ad98-788afd9779c2" (UID: "f3cbace6-6f72-4943-ad98-788afd9779c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:39:29.582412 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.582214    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45" event={"ID":"f3cbace6-6f72-4943-ad98-788afd9779c2","Type":"ContainerDied","Data":"0b78c0b09f734d70fe8a4b636c406a2c6b5d34fec7c613ea1819b26d6bded89b"}
Jul 03 14:39:29.583612 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.583578    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3cbace6-6f72-4943-ad98-788afd9779c2" (UID: "f3cbace6-6f72-4943-ad98-788afd9779c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:39:29.583699 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.583611    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw" (OuterVolumeSpecName: "kube-api-access-ww8kw") pod "f3cbace6-6f72-4943-ad98-788afd9779c2" (UID: "f3cbace6-6f72-4943-ad98-788afd9779c2"). InnerVolumeSpecName "kube-api-access-ww8kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:29.599774 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.599757    2561 scope.go:117] "RemoveContainer" containerID="66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"
Jul 03 14:39:29.600044 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:39:29.600028    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d\": container with ID starting with 66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d not found: ID does not exist" containerID="66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"
Jul 03 14:39:29.600110 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.600055    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d"} err="failed to get container status \"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d\": rpc error: code = NotFound desc = could not find container \"66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d\": container with ID starting with 66092ce57673456b326f60a39e8d162eff38b533450a0c0ca1deab6d20e0e32d not found: ID does not exist"
Jul 03 14:39:29.682096 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.682060    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww8kw\" (UniqueName: \"kubernetes.io/projected/f3cbace6-6f72-4943-ad98-788afd9779c2-kube-api-access-ww8kw\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:29.682096 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.682090    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3cbace6-6f72-4943-ad98-788afd9779c2-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:29.682096 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.682100    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3cbace6-6f72-4943-ad98-788afd9779c2-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:29.905895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.905868    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:29.911102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:29.911080    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-ea235c7d-6f6697fbc9-tbh45"]
Jul 03 14:39:30.191025 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:30.190946    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3cbace6-6f72-4943-ad98-788afd9779c2" path="/var/lib/kubelet/pods/f3cbace6-6f72-4943-ad98-788afd9779c2/volumes"
Jul 03 14:39:34.616118 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:34.616089    2561 generic.go:356] "Generic (PLEG): container finished" podID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" containerID="e984d62de12219dec6427c9ec279de1635669c857cb2f1fa731f17187b5d355d" exitCode=0
Jul 03 14:39:34.616500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:34.616163    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" event={"ID":"be4ec242-31c5-424e-9c0c-7aa638f8b9cc","Type":"ContainerDied","Data":"e984d62de12219dec6427c9ec279de1635669c857cb2f1fa731f17187b5d355d"}
Jul 03 14:39:35.135595 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.135571    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:35.230405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230376    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230414    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230441    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230460    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230482    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230553 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230542    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rps79\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230581    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230628    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230679    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket\") pod \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\" (UID: \"be4ec242-31c5-424e-9c0c-7aa638f8b9cc\") "
Jul 03 14:39:35.230812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230737    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:35.231025 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.230978    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.231082 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.231049    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:35.231198 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.231158    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:35.231268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.231240    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:39:35.231488 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.231385    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:35.232827 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.232803    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:39:35.232827 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.232812    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:35.233298 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.233275    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:39:35.233298 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.233289    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79" (OuterVolumeSpecName: "kube-api-access-rps79") pod "be4ec242-31c5-424e-9c0c-7aa638f8b9cc" (UID: "be4ec242-31c5-424e-9c0c-7aa638f8b9cc"). InnerVolumeSpecName "kube-api-access-rps79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331804    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rps79\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-kube-api-access-rps79\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331827    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331838    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331846    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331855    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.331862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331863    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.332115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331871    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.332115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.331878    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be4ec242-31c5-424e-9c0c-7aa638f8b9cc-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:39:35.495307 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.495273    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:39:35.622727 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.622704    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"
Jul 03 14:39:35.623144 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.622745    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv" event={"ID":"be4ec242-31c5-424e-9c0c-7aa638f8b9cc","Type":"ContainerDied","Data":"7a565e0f0f204add082f7b2db520c9f065abef0cc3429e2f2484669f7846c412"}
Jul 03 14:39:35.623144 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.622809    2561 scope.go:117] "RemoveContainer" containerID="e984d62de12219dec6427c9ec279de1635669c857cb2f1fa731f17187b5d355d"
Jul 03 14:39:35.648910 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.648889    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:35.651088 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:35.651070    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ea235c7d-openshift-default-77b9b4bcc9-k96rv"]
Jul 03 14:39:36.189976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:39:36.189947    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" path="/var/lib/kubelet/pods/be4ec242-31c5-424e-9c0c-7aa638f8b9cc/volumes"
Jul 03 14:40:59.408271 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:40:59.408238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:41:08.216273 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:08.216233    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:41:13.065922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.065886    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:13.066347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066259    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" containerName="istio-proxy"
Jul 03 14:41:13.066347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066271    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" containerName="istio-proxy"
Jul 03 14:41:13.066347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066284    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cbace6-6f72-4943-ad98-788afd9779c2" containerName="maas-api"
Jul 03 14:41:13.066347 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066289    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cbace6-6f72-4943-ad98-788afd9779c2" containerName="maas-api"
Jul 03 14:41:13.066480 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066353    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4ec242-31c5-424e-9c0c-7aa638f8b9cc" containerName="istio-proxy"
Jul 03 14:41:13.066480 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.066363    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cbace6-6f72-4943-ad98-788afd9779c2" containerName="maas-api"
Jul 03 14:41:13.069357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.069338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.072242 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.072223    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-40eb39c1-openshift-default-dockercfg-7w9vx\""
Jul 03 14:41:13.164144 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164107    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2wnn\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164221    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164322    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164357    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164374    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.164524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.164400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.174303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.174275    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:13.265495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265443    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265495 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265545    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265575    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265685    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2wnn\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.265770 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265785    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.265990    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.266077    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.266096    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266385 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.266366    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.266778 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.266752    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.267904 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.267877    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.268162 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.268144    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.277498 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.277470    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.279468 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.279446    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2wnn\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn\") pod \"e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") " pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.379008 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.378971    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:13.519122 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.519094    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:13.521678 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:41:13.521647    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72aca84_7150_4d95_8f00_f8f43a265b93.slice/crio-a4f23c9c51105367086858c0f1fbf9619e42f724698dd233f5e66156aac65de7 WatchSource:0}: Error finding container a4f23c9c51105367086858c0f1fbf9619e42f724698dd233f5e66156aac65de7: Status 404 returned error can't find the container with id a4f23c9c51105367086858c0f1fbf9619e42f724698dd233f5e66156aac65de7
Jul 03 14:41:13.523691 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.523661    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:13.523753 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.523727    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:13.523790 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:13.523755    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:14.075421 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:14.075377    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerStarted","Data":"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"}
Jul 03 14:41:14.075776 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:14.075439    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerStarted","Data":"a4f23c9c51105367086858c0f1fbf9619e42f724698dd233f5e66156aac65de7"}
Jul 03 14:41:14.100608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:14.100557    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" podStartSLOduration=1.100537503 podStartE2EDuration="1.100537503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:41:14.098724453 +0000 UTC m=+1776.512620940" watchObservedRunningTime="2026-07-03 14:41:14.100537503 +0000 UTC m=+1776.514433953"
Jul 03 14:41:14.380008 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:14.379970    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:14.384702 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:14.384679    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:15.080228 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:15.080196    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:15.081142 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:15.081121    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:16.192777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.192750    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:16.196251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.196230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.201886 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.201861    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:41:16.202011 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.201905    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-40eb39c1-serving-cert\""
Jul 03 14:41:16.209460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.209440    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:16.291988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.291952    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttfj\" (UniqueName: \"kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.291988 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.291991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.292289 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.292025    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.393147 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.393115    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.393329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.393173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.393329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.393285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttfj\" (UniqueName: \"kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.393867 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.393845    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.396021 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.396000    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.403230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.403205    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttfj\" (UniqueName: \"kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj\") pod \"maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") " pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.507972 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.507904    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:16.651321 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:16.651298    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:16.653746 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:41:16.653719    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258c6f7c_d72c_4eb7_8b3c_84d764818a94.slice/crio-9b7c46c3c1a17c7bda0be7cc9d07aa28e85289e0688feec41d49e5012f650e72 WatchSource:0}: Error finding container 9b7c46c3c1a17c7bda0be7cc9d07aa28e85289e0688feec41d49e5012f650e72: Status 404 returned error can't find the container with id 9b7c46c3c1a17c7bda0be7cc9d07aa28e85289e0688feec41d49e5012f650e72
Jul 03 14:41:17.090539 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:17.090505    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" event={"ID":"258c6f7c-d72c-4eb7-8b3c-84d764818a94","Type":"ContainerStarted","Data":"9b7c46c3c1a17c7bda0be7cc9d07aa28e85289e0688feec41d49e5012f650e72"}
Jul 03 14:41:18.096924 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:18.096886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" event={"ID":"258c6f7c-d72c-4eb7-8b3c-84d764818a94","Type":"ContainerStarted","Data":"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"}
Jul 03 14:41:18.097394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:18.096992    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:18.116044 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:18.115998    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" podStartSLOduration=1.59561618 podStartE2EDuration="2.115984086s" totalImagesPullingTime="520.367906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:41:18.115228729 +0000 UTC m=+1780.529125177" watchObservedRunningTime="2026-07-03 14:41:18.115984086 +0000 UTC m=+1780.529880539"
Jul 03 14:41:22.899173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:22.899139    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:41:24.107727 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:24.107700    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:28.577825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:28.577713    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:28.578242 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:28.577953    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" podUID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" containerName="maas-api" containerID="cri-o://d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910" gracePeriod=30
Jul 03 14:41:28.842094 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:28.842068    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:29.013839 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.013793    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zttfj\" (UniqueName: \"kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj\") pod \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") "
Jul 03 14:41:29.013839 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.013841    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca\") pod \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") "
Jul 03 14:41:29.014105 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.013914    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls\") pod \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\" (UID: \"258c6f7c-d72c-4eb7-8b3c-84d764818a94\") "
Jul 03 14:41:29.014249 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.014211    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "258c6f7c-d72c-4eb7-8b3c-84d764818a94" (UID: "258c6f7c-d72c-4eb7-8b3c-84d764818a94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:41:29.016231 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.016170    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj" (OuterVolumeSpecName: "kube-api-access-zttfj") pod "258c6f7c-d72c-4eb7-8b3c-84d764818a94" (UID: "258c6f7c-d72c-4eb7-8b3c-84d764818a94"). InnerVolumeSpecName "kube-api-access-zttfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:41:29.016231 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.016211    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "258c6f7c-d72c-4eb7-8b3c-84d764818a94" (UID: "258c6f7c-d72c-4eb7-8b3c-84d764818a94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:41:29.115389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.115300    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zttfj\" (UniqueName: \"kubernetes.io/projected/258c6f7c-d72c-4eb7-8b3c-84d764818a94-kube-api-access-zttfj\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:29.115389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.115338    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/258c6f7c-d72c-4eb7-8b3c-84d764818a94-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:29.115389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.115353    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/258c6f7c-d72c-4eb7-8b3c-84d764818a94-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:29.150573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.150538    2561 generic.go:356] "Generic (PLEG): container finished" podID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" containerID="d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910" exitCode=0
Jul 03 14:41:29.150748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.150594    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"
Jul 03 14:41:29.150748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.150621    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" event={"ID":"258c6f7c-d72c-4eb7-8b3c-84d764818a94","Type":"ContainerDied","Data":"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"}
Jul 03 14:41:29.150748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.150671    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn" event={"ID":"258c6f7c-d72c-4eb7-8b3c-84d764818a94","Type":"ContainerDied","Data":"9b7c46c3c1a17c7bda0be7cc9d07aa28e85289e0688feec41d49e5012f650e72"}
Jul 03 14:41:29.150748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.150697    2561 scope.go:117] "RemoveContainer" containerID="d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"
Jul 03 14:41:29.160784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.160762    2561 scope.go:117] "RemoveContainer" containerID="d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"
Jul 03 14:41:29.161088 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:41:29.161062    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910\": container with ID starting with d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910 not found: ID does not exist" containerID="d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"
Jul 03 14:41:29.161214 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.161092    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910"} err="failed to get container status \"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910\": rpc error: code = NotFound desc = could not find container \"d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910\": container with ID starting with d11d5e349525f1c6ce7173999de44bf2dfe165407c6f5ba4b5988f8010046910 not found: ID does not exist"
Jul 03 14:41:29.177812 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.177780    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:29.185493 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:29.185468    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-40eb39c1-747cc7b9bb-7rmsn"]
Jul 03 14:41:30.190456 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:30.190429    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" path="/var/lib/kubelet/pods/258c6f7c-d72c-4eb7-8b3c-84d764818a94/volumes"
Jul 03 14:41:30.959108 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:30.958363    2561 patch_prober.go:28] interesting pod/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:56884->10.133.0.36:15021: read: connection reset by peer" start-of-body=
Jul 03 14:41:30.959108 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:30.958444    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:56884->10.133.0.36:15021: read: connection reset by peer"
Jul 03 14:41:31.164840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.164770    2561 generic.go:356] "Generic (PLEG): container finished" podID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerID="bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357" exitCode=0
Jul 03 14:41:31.164983 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.164839    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerDied","Data":"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"}
Jul 03 14:41:31.165279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.165265    2561 scope.go:117] "RemoveContainer" containerID="bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"
Jul 03 14:41:31.166312 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.166286    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:31.166389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.166372    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:31.166430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.166420    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:31.200522 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:31.200487    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:41:32.173870 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:32.173838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerStarted","Data":"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"}
Jul 03 14:41:32.380002 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:32.379975    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:32.383226 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:32.383206    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:33.178821 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:33.178795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:33.179698 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:33.179679    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:35.111586 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:35.111555    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:35.187405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:35.187375    2561 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" secret="" err="secret \"e2e-mt-40eb39c1-openshift-default-dockercfg-7w9vx\" not found"
Jul 03 14:41:36.191351 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:36.191310    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy" containerID="cri-o://b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae" gracePeriod=30
Jul 03 14:41:38.170844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:38.170814    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:41:38.177154 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:38.177130    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:41:41.444942 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.444917    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:41.529704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529608    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.529704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529673    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.529919 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529805    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.529919 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529834    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.529919 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529881    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:41:41.529919 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529897    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.530098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529942    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:41:41.530098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529971    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2wnn\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.530098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.529996    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.530098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530019    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.530098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530047    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo\") pod \"b72aca84-7150-4d95-8f00-f8f43a265b93\" (UID: \"b72aca84-7150-4d95-8f00-f8f43a265b93\") "
Jul 03 14:41:41.530370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530349    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:41:41.530436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530368    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:41:41.530436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530413    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.530436 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530432    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.530994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.530966    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data" (OuterVolumeSpecName: "istio-data") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:41:41.532209 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.532161    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token" (OuterVolumeSpecName: "istio-token") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:41:41.532315 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.532249    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:41:41.532315 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.532269    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:41:41.532409 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.532329    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn" (OuterVolumeSpecName: "kube-api-access-n2wnn") pod "b72aca84-7150-4d95-8f00-f8f43a265b93" (UID: "b72aca84-7150-4d95-8f00-f8f43a265b93"). InnerVolumeSpecName "kube-api-access-n2wnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:41:41.631097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631064    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631094    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631103    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631282 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631111    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2wnn\" (UniqueName: \"kubernetes.io/projected/b72aca84-7150-4d95-8f00-f8f43a265b93-kube-api-access-n2wnn\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631282 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631120    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72aca84-7150-4d95-8f00-f8f43a265b93-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631282 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631128    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72aca84-7150-4d95-8f00-f8f43a265b93-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:41.631282 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:41.631135    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72aca84-7150-4d95-8f00-f8f43a265b93-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:41:42.219446 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.219413    2561 generic.go:356] "Generic (PLEG): container finished" podID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerID="b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae" exitCode=0
Jul 03 14:41:42.219572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.219459    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"
Jul 03 14:41:42.219572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.219492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerDied","Data":"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"}
Jul 03 14:41:42.219572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.219541    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95" event={"ID":"b72aca84-7150-4d95-8f00-f8f43a265b93","Type":"ContainerDied","Data":"a4f23c9c51105367086858c0f1fbf9619e42f724698dd233f5e66156aac65de7"}
Jul 03 14:41:42.219572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.219562    2561 scope.go:117] "RemoveContainer" containerID="b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"
Jul 03 14:41:42.228176 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.228161    2561 scope.go:117] "RemoveContainer" containerID="bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"
Jul 03 14:41:42.235528 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.235512    2561 scope.go:117] "RemoveContainer" containerID="b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"
Jul 03 14:41:42.235775 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:41:42.235757    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae\": container with ID starting with b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae not found: ID does not exist" containerID="b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"
Jul 03 14:41:42.235848 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.235786    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae"} err="failed to get container status \"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae\": rpc error: code = NotFound desc = could not find container \"b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae\": container with ID starting with b0d1a591aad8373cfdaf5414aeea4e270fdc193d28c37d4e4a4c0785301d01ae not found: ID does not exist"
Jul 03 14:41:42.235848 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.235808    2561 scope.go:117] "RemoveContainer" containerID="bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"
Jul 03 14:41:42.236079 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:41:42.236060    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357\": container with ID starting with bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357 not found: ID does not exist" containerID="bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"
Jul 03 14:41:42.236125 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.236085    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357"} err="failed to get container status \"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357\": rpc error: code = NotFound desc = could not find container \"bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357\": container with ID starting with bc56a1ce431a87d0f052ca5d06591fa9b1be43881925276d0f14ae1e79f22357 not found: ID does not exist"
Jul 03 14:41:42.241642 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.241622    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:42.243972 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:42.243951    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-40eb39c1-openshift-default-79bfb45555-k2m95"]
Jul 03 14:41:44.190399 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:44.190351    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" path="/var/lib/kubelet/pods/b72aca84-7150-4d95-8f00-f8f43a265b93/volumes"
Jul 03 14:41:55.692071 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692033    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:41:55.692602 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692582    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692605    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692618    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692626    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692647    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" containerName="maas-api"
Jul 03 14:41:55.692676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692656    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" containerName="maas-api"
Jul 03 14:41:55.692922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692745    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692760    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72aca84-7150-4d95-8f00-f8f43a265b93" containerName="istio-proxy"
Jul 03 14:41:55.692922 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.692771    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="258c6f7c-d72c-4eb7-8b3c-84d764818a94" containerName="maas-api"
Jul 03 14:41:55.697450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.697430    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.707198 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.707156    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fb67b703-openshift-default-dockercfg-hzc8x\""
Jul 03 14:41:55.725035 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.725012    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:41:55.748377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748350    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748480    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748506    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzbk\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748641    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.748858 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.748729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849316    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849352    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849401    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849709    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849790    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.849958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.849899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzbk\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.850337 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.850242    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.850337 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.850286    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.850464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.850439    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.852526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.852504    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.852631 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.852615    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.877793 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.877771    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzbk\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:55.879740 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:55.879724    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token\") pod \"e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") " pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:56.007666 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.007583    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:56.151784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.151733    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:41:56.156493 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:41:56.155242    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7978d6c8_787c_43ed_bc4d_7e575fe01658.slice/crio-cdbfd277df09576d025703d68abe1d727ff811a19c35302cb149fd055f5b327f WatchSource:0}: Error finding container cdbfd277df09576d025703d68abe1d727ff811a19c35302cb149fd055f5b327f: Status 404 returned error can't find the container with id cdbfd277df09576d025703d68abe1d727ff811a19c35302cb149fd055f5b327f
Jul 03 14:41:56.158444 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.158398    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:56.158573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.158479    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:56.158573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.158529    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:41:56.284859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.284761    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerStarted","Data":"c76c43360278f663aa35e2ab81b0a234d4a64450186e4cf3abe9705a3c824d82"}
Jul 03 14:41:56.284859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.284822    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerStarted","Data":"cdbfd277df09576d025703d68abe1d727ff811a19c35302cb149fd055f5b327f"}
Jul 03 14:41:56.325959 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:56.325909    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podStartSLOduration=1.325893429 podStartE2EDuration="1.325893429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:41:56.324147988 +0000 UTC m=+1818.738044450" watchObservedRunningTime="2026-07-03 14:41:56.325893429 +0000 UTC m=+1818.739789879"
Jul 03 14:41:57.007851 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:57.007811    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:57.012618 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:57.012597    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:57.290121 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:57.290035    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:57.290856 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:57.290837    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:41:58.743699 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.743663    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:41:58.747113 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.747097    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.768247 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.768224    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-fb67b703-serving-cert\""
Jul 03 14:41:58.768365 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.768257    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:41:58.770160 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.770141    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:41:58.877749 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.877717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5628\" (UniqueName: \"kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.877924 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.877759    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.877924 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.877819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.979146 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.979106    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5628\" (UniqueName: \"kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.979146 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.979150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.979359 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.979175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.979801 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.979771    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:58.981873 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:58.981841    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:59.016650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:59.016581    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5628\" (UniqueName: \"kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628\") pod \"maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") " pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:59.059027 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:59.059001    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:41:59.224784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:59.224759    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:41:59.226534 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:41:59.226500    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0abfabc_b607_440b_8e9e_82269d107229.slice/crio-7e4164f7443db386ac6697059b325594871b5cfca2f6ce85e9529ac5b96e099f WatchSource:0}: Error finding container 7e4164f7443db386ac6697059b325594871b5cfca2f6ce85e9529ac5b96e099f: Status 404 returned error can't find the container with id 7e4164f7443db386ac6697059b325594871b5cfca2f6ce85e9529ac5b96e099f
Jul 03 14:41:59.301559 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:41:59.301479    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" event={"ID":"d0abfabc-b607-440b-8e9e-82269d107229","Type":"ContainerStarted","Data":"7e4164f7443db386ac6697059b325594871b5cfca2f6ce85e9529ac5b96e099f"}
Jul 03 14:42:00.307807 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:00.307774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" event={"ID":"d0abfabc-b607-440b-8e9e-82269d107229","Type":"ContainerStarted","Data":"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"}
Jul 03 14:42:00.308259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:00.307907    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:42:00.334797 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:00.334748    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" podStartSLOduration=1.96764456 podStartE2EDuration="2.334734919s" totalImagesPullingTime="367.090359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:42:00.333071364 +0000 UTC m=+1822.746967816" watchObservedRunningTime="2026-07-03 14:42:00.334734919 +0000 UTC m=+1822.748631369"
Jul 03 14:42:01.814558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:01.814525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:42:06.318778 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:06.318745    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:42:07.899148 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.899112    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:42:07.904125 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.904105    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:07.913145 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.913120    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fb67b703\"/\"openshift-service-ca.crt\""
Jul 03 14:42:07.913308 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.913120    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fb67b703\"/\"default-dockercfg-qbs2r\""
Jul 03 14:42:07.913308 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.913174    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fb67b703\"/\"e2e-lifecycle-model-fb67b703-kserve-self-signed-certs\""
Jul 03 14:42:07.913308 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.913225    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fb67b703\"/\"kube-root-ca.crt\""
Jul 03 14:42:07.921785 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:07.921760    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:42:08.071319 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071278    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.071476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071340    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.071476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071389    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.071568 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071470    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.071568 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071524    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.071639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.071579    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rws\" (UniqueName: \"kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173151 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173070    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173151 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173133    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rws\" (UniqueName: \"kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173233    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173377 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173524 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173419    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173568    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173644 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173603    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.173729 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.173707    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.175886 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.175861    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.175994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.175973    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.191269 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.191235    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rws\" (UniqueName: \"kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws\") pod \"e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") " pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.213456 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.213423    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:08.370534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:08.370510    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:42:08.373898 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:42:08.373872    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e460cd7_92f5_4778_bc13_cd97cdfba1f2.slice/crio-ced3cd17923324e600706bf40c0f3e34c0ed85eccc208c78d6dea3475405ced4 WatchSource:0}: Error finding container ced3cd17923324e600706bf40c0f3e34c0ed85eccc208c78d6dea3475405ced4: Status 404 returned error can't find the container with id ced3cd17923324e600706bf40c0f3e34c0ed85eccc208c78d6dea3475405ced4
Jul 03 14:42:09.354525 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:09.354478    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" event={"ID":"6e460cd7-92f5-4778-bc13-cd97cdfba1f2","Type":"ContainerStarted","Data":"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"}
Jul 03 14:42:09.354963 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:09.354534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" event={"ID":"6e460cd7-92f5-4778-bc13-cd97cdfba1f2","Type":"ContainerStarted","Data":"ced3cd17923324e600706bf40c0f3e34c0ed85eccc208c78d6dea3475405ced4"}
Jul 03 14:42:09.382802 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:09.382742    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" podStartSLOduration=2.382723736 podStartE2EDuration="2.382723736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:42:09.382072456 +0000 UTC m=+1831.795968907" watchObservedRunningTime="2026-07-03 14:42:09.382723736 +0000 UTC m=+1831.796620164"
Jul 03 14:42:10.360608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.360575    2561 generic.go:356] "Generic (PLEG): container finished" podID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerID="c76c43360278f663aa35e2ab81b0a234d4a64450186e4cf3abe9705a3c824d82" exitCode=0
Jul 03 14:42:10.361024 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.360645    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerDied","Data":"c76c43360278f663aa35e2ab81b0a234d4a64450186e4cf3abe9705a3c824d82"}
Jul 03 14:42:10.361077 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.361063    2561 scope.go:117] "RemoveContainer" containerID="c76c43360278f663aa35e2ab81b0a234d4a64450186e4cf3abe9705a3c824d82"
Jul 03 14:42:10.362003 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.361977    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:42:10.362123 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.362039    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:42:10.362123 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:10.362071    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:42:11.372268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:11.372236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerStarted","Data":"093dca184d2da78629e6aae1b8018daece684571fc51917bf29ea8e081e73a6b"}
Jul 03 14:42:11.438403 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:11.438375    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:42:12.008672 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:12.008637    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:42:12.009768 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:12.009739    2561 patch_prober.go:28] interesting pod/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 03 14:42:12.009900 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:12.009791    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 03 14:42:12.290744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:12.290656    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:42:13.008743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:13.008705    2561 patch_prober.go:28] interesting pod/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 03 14:42:13.009146 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:13.008761    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 03 14:42:14.008157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:14.008119    2561 patch_prober.go:28] interesting pod/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 03 14:42:14.008359 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:14.008176    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 03 14:42:15.008843 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:15.008808    2561 patch_prober.go:28] interesting pod/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 03 14:42:15.009348 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:15.008863    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 03 14:42:16.011986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:16.011957    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:42:16.012925 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:16.012905    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:42:18.214149 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:18.214121    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:18.214535 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:18.214193    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:18.226576 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:18.226555    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:18.416631 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:18.416607    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:42:38.317973 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:38.317942    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:42:41.654062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.654028    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:42:41.654542 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.654291    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" podUID="d0abfabc-b607-440b-8e9e-82269d107229" containerName="maas-api" containerID="cri-o://a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1" gracePeriod=30
Jul 03 14:42:41.895739 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.895715    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:42:41.961509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.961433    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca\") pod \"d0abfabc-b607-440b-8e9e-82269d107229\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") "
Jul 03 14:42:41.961653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.961542    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5628\" (UniqueName: \"kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628\") pod \"d0abfabc-b607-440b-8e9e-82269d107229\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") "
Jul 03 14:42:41.961653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.961593    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls\") pod \"d0abfabc-b607-440b-8e9e-82269d107229\" (UID: \"d0abfabc-b607-440b-8e9e-82269d107229\") "
Jul 03 14:42:41.961844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.961823    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0abfabc-b607-440b-8e9e-82269d107229" (UID: "d0abfabc-b607-440b-8e9e-82269d107229"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:42:41.961899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.961886    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0abfabc-b607-440b-8e9e-82269d107229-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:42:41.963715 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.963692    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628" (OuterVolumeSpecName: "kube-api-access-s5628") pod "d0abfabc-b607-440b-8e9e-82269d107229" (UID: "d0abfabc-b607-440b-8e9e-82269d107229"). InnerVolumeSpecName "kube-api-access-s5628". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:42:41.963781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:41.963710    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0abfabc-b607-440b-8e9e-82269d107229" (UID: "d0abfabc-b607-440b-8e9e-82269d107229"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:42:42.063032 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.062984    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5628\" (UniqueName: \"kubernetes.io/projected/d0abfabc-b607-440b-8e9e-82269d107229-kube-api-access-s5628\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:42:42.063032 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.063024    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0abfabc-b607-440b-8e9e-82269d107229-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:42:42.523486 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.523447    2561 generic.go:356] "Generic (PLEG): container finished" podID="d0abfabc-b607-440b-8e9e-82269d107229" containerID="a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1" exitCode=0
Jul 03 14:42:42.523661 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.523502    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"
Jul 03 14:42:42.523661 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.523526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" event={"ID":"d0abfabc-b607-440b-8e9e-82269d107229","Type":"ContainerDied","Data":"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"}
Jul 03 14:42:42.523661 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.523579    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt" event={"ID":"d0abfabc-b607-440b-8e9e-82269d107229","Type":"ContainerDied","Data":"7e4164f7443db386ac6697059b325594871b5cfca2f6ce85e9529ac5b96e099f"}
Jul 03 14:42:42.523661 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.523602    2561 scope.go:117] "RemoveContainer" containerID="a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"
Jul 03 14:42:42.532159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.532139    2561 scope.go:117] "RemoveContainer" containerID="a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"
Jul 03 14:42:42.532425 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:42:42.532405    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1\": container with ID starting with a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1 not found: ID does not exist" containerID="a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"
Jul 03 14:42:42.532476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.532432    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1"} err="failed to get container status \"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1\": rpc error: code = NotFound desc = could not find container \"a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1\": container with ID starting with a6e6b8360104e824ee3f1051507fa825466ae4996b6cd5aeef681c0dc96887a1 not found: ID does not exist"
Jul 03 14:42:42.543408 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.543390    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:42:42.548857 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:42.548837    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-fb67b703-94c7d85d4-bghlt"]
Jul 03 14:42:44.192290 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:44.192249    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0abfabc-b607-440b-8e9e-82269d107229" path="/var/lib/kubelet/pods/d0abfabc-b607-440b-8e9e-82269d107229/volumes"
Jul 03 14:42:51.741405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:51.741372    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:42:52.607096 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:42:52.607062    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:43:06.761775 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:06.761660    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" podUID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" containerName="main" containerID="cri-o://48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994" gracePeriod=45
Jul 03 14:43:07.012748 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.012689    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:43:07.188077 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188032    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.188336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188101    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.188336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188136    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.188336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188173    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.188504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188363    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8rws\" (UniqueName: \"kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.188504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.188394    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm\") pod \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\" (UID: \"6e460cd7-92f5-4778-bc13-cd97cdfba1f2\") "
Jul 03 14:43:07.194233 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.194158    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home" (OuterVolumeSpecName: "home") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:07.194466 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.194311    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:07.194676 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.194361    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:07.194840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.194813    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm" (OuterVolumeSpecName: "dshm") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:07.196583 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.196561    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws" (OuterVolumeSpecName: "kube-api-access-q8rws") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "kube-api-access-q8rws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:43:07.196875 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.196857    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e460cd7-92f5-4778-bc13-cd97cdfba1f2" (UID: "6e460cd7-92f5-4778-bc13-cd97cdfba1f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:43:07.289566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289483    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8rws\" (UniqueName: \"kubernetes.io/projected/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-kube-api-access-q8rws\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.289566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289509    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-dshm\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.289755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289592    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-model-cache\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.289755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289730    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tls-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.289755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289747    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-home\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.289861 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.289760    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e460cd7-92f5-4778-bc13-cd97cdfba1f2-tmp-dir\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:07.654717 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.654683    2561 generic.go:356] "Generic (PLEG): container finished" podID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" containerID="48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994" exitCode=0
Jul 03 14:43:07.654905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.654736    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"
Jul 03 14:43:07.654905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.654758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" event={"ID":"6e460cd7-92f5-4778-bc13-cd97cdfba1f2","Type":"ContainerDied","Data":"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"}
Jul 03 14:43:07.654905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.654803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d" event={"ID":"6e460cd7-92f5-4778-bc13-cd97cdfba1f2","Type":"ContainerDied","Data":"ced3cd17923324e600706bf40c0f3e34c0ed85eccc208c78d6dea3475405ced4"}
Jul 03 14:43:07.654905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.654821    2561 scope.go:117] "RemoveContainer" containerID="48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"
Jul 03 14:43:07.663923 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.663906    2561 scope.go:117] "RemoveContainer" containerID="48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"
Jul 03 14:43:07.664167 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:43:07.664149    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994\": container with ID starting with 48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994 not found: ID does not exist" containerID="48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"
Jul 03 14:43:07.664243 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.664191    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994"} err="failed to get container status \"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994\": rpc error: code = NotFound desc = could not find container \"48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994\": container with ID starting with 48d7789ff1ac05128cd366078a5ae71a408afb010b564768d4133a46c857f994 not found: ID does not exist"
Jul 03 14:43:07.676898 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.676877    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:43:07.681431 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:07.681414    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fb67b703/e2e-lifecycle-model-fb67b703-kserve-57959776cf-x7d8d"]
Jul 03 14:43:08.192090 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:08.192060    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" path="/var/lib/kubelet/pods/6e460cd7-92f5-4778-bc13-cd97cdfba1f2/volumes"
Jul 03 14:43:31.499896 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:31.499823    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:43:35.694859 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:35.694813    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:43:35.695458 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:35.695025    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy" containerID="cri-o://093dca184d2da78629e6aae1b8018daece684571fc51917bf29ea8e081e73a6b" gracePeriod=30
Jul 03 14:43:40.813573 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:40.813525    2561 generic.go:356] "Generic (PLEG): container finished" podID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerID="093dca184d2da78629e6aae1b8018daece684571fc51917bf29ea8e081e73a6b" exitCode=0
Jul 03 14:43:40.813933 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:40.813600    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerDied","Data":"093dca184d2da78629e6aae1b8018daece684571fc51917bf29ea8e081e73a6b"}
Jul 03 14:43:40.813933 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:40.813659    2561 scope.go:117] "RemoveContainer" containerID="c76c43360278f663aa35e2ab81b0a234d4a64450186e4cf3abe9705a3c824d82"
Jul 03 14:43:40.950892 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:40.950872    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:43:41.085057 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085034    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085071    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085169    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085245 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085218    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085252    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tzbk\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085278    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085335    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket\") pod \"7978d6c8-787c-43ed-bc4d-7e575fe01658\" (UID: \"7978d6c8-787c-43ed-bc4d-7e575fe01658\") "
Jul 03 14:43:41.085423 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085408    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:41.085626 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085448    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:41.085626 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085604    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:43:41.085759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085738    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:41.085819 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085757    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7978d6c8-787c-43ed-bc4d-7e575fe01658-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.085819 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085777    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.085819 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.085790    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.086340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.086314    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data" (OuterVolumeSpecName: "istio-data") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:41.087593 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.087566    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk" (OuterVolumeSpecName: "kube-api-access-4tzbk") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "kube-api-access-4tzbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:43:41.087750 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.087732    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:43:41.088040 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.088020    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:43:41.088097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.088032    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token" (OuterVolumeSpecName: "istio-token") pod "7978d6c8-787c-43ed-bc4d-7e575fe01658" (UID: "7978d6c8-787c-43ed-bc4d-7e575fe01658"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:43:41.186292 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186268    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.186292 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186291    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.186450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186302    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tzbk\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-kube-api-access-4tzbk\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.186450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186310    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.186450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186319    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7978d6c8-787c-43ed-bc4d-7e575fe01658-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.186450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.186328    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7978d6c8-787c-43ed-bc4d-7e575fe01658-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:43:41.819095 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.819066    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"
Jul 03 14:43:41.819095 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.819086    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47" event={"ID":"7978d6c8-787c-43ed-bc4d-7e575fe01658","Type":"ContainerDied","Data":"cdbfd277df09576d025703d68abe1d727ff811a19c35302cb149fd055f5b327f"}
Jul 03 14:43:41.819593 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.819140    2561 scope.go:117] "RemoveContainer" containerID="093dca184d2da78629e6aae1b8018daece684571fc51917bf29ea8e081e73a6b"
Jul 03 14:43:41.842790 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.842766    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:43:41.848125 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:41.848103    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fb67b703-openshift-default-58fbdd75b7-xsd47"]
Jul 03 14:43:42.190425 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:42.190392    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" path="/var/lib/kubelet/pods/7978d6c8-787c-43ed-bc4d-7e575fe01658/volumes"
Jul 03 14:43:43.900221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:43.900173    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:43:53.994874 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:43:53.994839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:44:31.423143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423113    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:44:31.423687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423632    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:31.423687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423652    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:31.423687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423662    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" containerName="main"
Jul 03 14:44:31.423687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423670    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" containerName="main"
Jul 03 14:44:31.423687 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423682    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423691    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423710    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0abfabc-b607-440b-8e9e-82269d107229" containerName="maas-api"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423718    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0abfabc-b607-440b-8e9e-82269d107229" containerName="maas-api"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423810    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0abfabc-b607-440b-8e9e-82269d107229" containerName="maas-api"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423820    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e460cd7-92f5-4778-bc13-cd97cdfba1f2" containerName="main"
Jul 03 14:44:31.423949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.423830    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:31.427115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.427097    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.429666 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.429647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c1cb72-openshift-default-dockercfg-l5tvl\""
Jul 03 14:44:31.438238 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.438217    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:44:31.518808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518776    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.518978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518822    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhpmh\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.518978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.518978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.518978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.518978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.518976    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.519203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.519005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.519203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.519047    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.519203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.519077    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619715 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619679    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619771    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhpmh\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619797    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619852    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619866    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.619908 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.619902    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.620331 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.620292    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.620413 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.620326    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.620413 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.620382    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.620506 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.620438    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.620644 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.620619    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.622411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.622391    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.622720 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.622702    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.628712 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.628692    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.629622 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.629592    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhpmh\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh\") pod \"e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") " pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.737395 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.737327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:31.875712 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.875685    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:44:31.878038 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:44:31.878008    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea4fcf9_a4ca_4fd0_aff5_0d7658ae6d81.slice/crio-ff4f3da9bc2f33b302171397779fb4fef292f2c707146b54761afeb8268abd9d WatchSource:0}: Error finding container ff4f3da9bc2f33b302171397779fb4fef292f2c707146b54761afeb8268abd9d: Status 404 returned error can't find the container with id ff4f3da9bc2f33b302171397779fb4fef292f2c707146b54761afeb8268abd9d
Jul 03 14:44:31.880303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.880272    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:31.880388 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.880342    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:31.880388 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:31.880367    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:32.057203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:32.057078    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerStarted","Data":"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"}
Jul 03 14:44:32.057203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:32.057135    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerStarted","Data":"ff4f3da9bc2f33b302171397779fb4fef292f2c707146b54761afeb8268abd9d"}
Jul 03 14:44:32.078862 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:32.078799    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podStartSLOduration=1.078776468 podStartE2EDuration="1.078776468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:32.07674746 +0000 UTC m=+1974.490643915" watchObservedRunningTime="2026-07-03 14:44:32.078776468 +0000 UTC m=+1974.492672918"
Jul 03 14:44:32.737542 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:32.737506    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:32.742623 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:32.742599    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:33.062461 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:33.062382    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:33.063340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:33.063323    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:34.241849 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.241817    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:44:34.242335 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.242241    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7978d6c8-787c-43ed-bc4d-7e575fe01658" containerName="istio-proxy"
Jul 03 14:44:34.245320 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.245300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.248019 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.247997    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:44:34.248234 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.248214    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-c1cb72-serving-cert\""
Jul 03 14:44:34.256132 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.256097    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:44:34.344880 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.344851    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2bd6\" (UniqueName: \"kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.345069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.344920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.345069 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.344959    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.445790 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.445742    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2bd6\" (UniqueName: \"kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.445980 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.445860    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.445980 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.445902    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.446782 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.446751    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.449357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.449332    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.460331 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.460311    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2bd6\" (UniqueName: \"kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6\") pod \"maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") " pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.557366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.557295    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:34.692285 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.692257    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:44:34.696476 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:44:34.696444    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbda208f_2e8e_43c7_9674_14c8e8fc6612.slice/crio-002f0f8f6fecd84cf98b8d36f725da993b9d79beb14d0b394eba75750b22cd85 WatchSource:0}: Error finding container 002f0f8f6fecd84cf98b8d36f725da993b9d79beb14d0b394eba75750b22cd85: Status 404 returned error can't find the container with id 002f0f8f6fecd84cf98b8d36f725da993b9d79beb14d0b394eba75750b22cd85
Jul 03 14:44:34.698073 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:34.698056    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:44:35.072913 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:35.072875    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" event={"ID":"fbda208f-2e8e-43c7-9674-14c8e8fc6612","Type":"ContainerStarted","Data":"002f0f8f6fecd84cf98b8d36f725da993b9d79beb14d0b394eba75750b22cd85"}
Jul 03 14:44:36.079529 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:36.079491    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" event={"ID":"fbda208f-2e8e-43c7-9674-14c8e8fc6612","Type":"ContainerStarted","Data":"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"}
Jul 03 14:44:36.080045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:36.079695    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:36.097778 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:36.097723    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" podStartSLOduration=1.523922324 podStartE2EDuration="2.097706687s" totalImagesPullingTime="573.784363ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:36.096388356 +0000 UTC m=+1978.510284819" watchObservedRunningTime="2026-07-03 14:44:36.097706687 +0000 UTC m=+1978.511603136"
Jul 03 14:44:39.374780 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.374750    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:44:39.379599 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.379578    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.382420 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.382394    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e58844-openshift-default-dockercfg-6g76r\""
Jul 03 14:44:39.397328 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.397308    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:44:39.492714 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492676    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.492895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492725    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.492895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492783    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brk9f\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.492895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492831    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.492895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492848    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.492895 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492866    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.493070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492940    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.493070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492957    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.493070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.492985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.593986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.593953    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.593992    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594017    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594147    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594225 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594236    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594290    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brk9f\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594412    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594508    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594733 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594564    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594733 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594706    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.594844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.594828    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.596458 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.596437    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.596868 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.596847    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.603540 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.603516    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brk9f\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.603654 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.603634    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token\") pod \"e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") " pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.690378 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.690297    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:39.829714 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.829690    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:44:39.831611 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:44:39.831582    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1597e6ca_7729_4741_ada8_3d464bb91e47.slice/crio-d4168beafa23a8952b6d43d720b169aff86c75ad8c9c683a2b154c1e89a3d520 WatchSource:0}: Error finding container d4168beafa23a8952b6d43d720b169aff86c75ad8c9c683a2b154c1e89a3d520: Status 404 returned error can't find the container with id d4168beafa23a8952b6d43d720b169aff86c75ad8c9c683a2b154c1e89a3d520
Jul 03 14:44:39.833853 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.833818    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:39.833928 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.833911    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:39.833980 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:39.833963    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:40.005974 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.005881    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:44:40.108696 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.108658    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerStarted","Data":"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"}
Jul 03 14:44:40.108881 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.108736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerStarted","Data":"d4168beafa23a8952b6d43d720b169aff86c75ad8c9c683a2b154c1e89a3d520"}
Jul 03 14:44:40.141966 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.141915    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podStartSLOduration=1.141899804 podStartE2EDuration="1.141899804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:40.138820261 +0000 UTC m=+1982.552716711" watchObservedRunningTime="2026-07-03 14:44:40.141899804 +0000 UTC m=+1982.555796253"
Jul 03 14:44:40.690671 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.690632    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:40.695608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:40.695582    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:41.113712 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:41.113676    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:41.114704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:41.114684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:42.090356 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.090330    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:44:42.334785 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.334756    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:44:42.338455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.338425    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.341357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.341294    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-e58844-serving-cert\""
Jul 03 14:44:42.358277 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.354516    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:44:42.518964 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.518930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.519150 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.518988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.519150 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.519014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvr7\" (UniqueName: \"kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.619607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.619517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.619607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.619563    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.619607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.619581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvr7\" (UniqueName: \"kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.619874 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:44:42.619676    2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-e58844-serving-cert: secret "maas-api-e2e-inf-b-e58844-serving-cert" not found
Jul 03 14:44:42.619874 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:44:42.619741    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls podName:13bbdb98-746a-4162-bbf3-651b2b4c1abe nodeName:}" failed. No retries permitted until 2026-07-03 14:44:43.119723507 +0000 UTC m=+1985.533619942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls") pod "maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" (UID: "13bbdb98-746a-4162-bbf3-651b2b4c1abe") : secret "maas-api-e2e-inf-b-e58844-serving-cert" not found
Jul 03 14:44:42.620328 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.620306    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:42.636742 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:42.636712    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvr7\" (UniqueName: \"kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:43.124394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:43.124357    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:43.127362 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:43.127334    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") pod \"maas-api-e2e-inf-b-e58844-869858cf74-vnzsw\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") " pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:43.262723 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:43.262693    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:43.428220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:43.428194    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:44:43.430350 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:44:43.430319    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13bbdb98_746a_4162_bbf3_651b2b4c1abe.slice/crio-0db189ba626419df82629f3853bf9475b992e85eeb783afb878396c0b66abe58 WatchSource:0}: Error finding container 0db189ba626419df82629f3853bf9475b992e85eeb783afb878396c0b66abe58: Status 404 returned error can't find the container with id 0db189ba626419df82629f3853bf9475b992e85eeb783afb878396c0b66abe58
Jul 03 14:44:44.129941 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:44.129908    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" event={"ID":"13bbdb98-746a-4162-bbf3-651b2b4c1abe","Type":"ContainerStarted","Data":"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"}
Jul 03 14:44:44.130336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:44.129960    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" event={"ID":"13bbdb98-746a-4162-bbf3-651b2b4c1abe","Type":"ContainerStarted","Data":"0db189ba626419df82629f3853bf9475b992e85eeb783afb878396c0b66abe58"}
Jul 03 14:44:44.130336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:44.130031    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:44.152985 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:44.152880    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" podStartSLOduration=1.696339954 podStartE2EDuration="2.152860463s" totalImagesPullingTime="456.520509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:44.152038133 +0000 UTC m=+1986.565934597" watchObservedRunningTime="2026-07-03 14:44:44.152860463 +0000 UTC m=+1986.566756914"
Jul 03 14:44:48.427500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:48.426296    2561 patch_prober.go:28] interesting pod/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:52954->10.133.0.41:15021: read: connection reset by peer" start-of-body=
Jul 03 14:44:48.427500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:48.426373    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:52954->10.133.0.41:15021: read: connection reset by peer"
Jul 03 14:44:49.163301 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.163259    2561 generic.go:356] "Generic (PLEG): container finished" podID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerID="20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531" exitCode=0
Jul 03 14:44:49.163512 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.163319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerDied","Data":"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"}
Jul 03 14:44:49.163851 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.163825    2561 scope.go:117] "RemoveContainer" containerID="20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"
Jul 03 14:44:49.165387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.165347    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:49.165492 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.165445    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:49.165566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:49.165494    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:50.140519 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:50.140487    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:44:50.178509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:50.178465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerStarted","Data":"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"}
Jul 03 14:44:50.737640 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:50.737607    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:50.737907 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:50.737881    2561 patch_prober.go:28] interesting pod/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 03 14:44:50.737965 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:50.737941    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 03 14:44:51.737984 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:51.737944    2561 patch_prober.go:28] interesting pod/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 03 14:44:51.738394 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:51.738004    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 03 14:44:52.415398 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:52.415358    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:44:52.738168 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:52.738081    2561 patch_prober.go:28] interesting pod/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 03 14:44:52.738557 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:52.738161    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 03 14:44:53.738009 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:53.737966    2561 patch_prober.go:28] interesting pod/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 03 14:44:53.738153 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:53.738067    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 03 14:44:54.200294 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.200264    2561 generic.go:356] "Generic (PLEG): container finished" podID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerID="db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be" exitCode=0
Jul 03 14:44:54.200760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.200331    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerDied","Data":"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"}
Jul 03 14:44:54.200760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.200705    2561 scope.go:117] "RemoveContainer" containerID="db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"
Jul 03 14:44:54.201737 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.201706    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:54.201829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.201775    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:54.201829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.201806    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:44:54.741576 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.741538    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:54.741840 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.741817    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:54.742601 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:54.742582    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:44:55.207695 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:55.207663    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerStarted","Data":"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"}
Jul 03 14:44:55.690744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:55.690711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:55.691998 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:55.691973    2561 patch_prober.go:28] interesting pod/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 03 14:44:55.692112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:55.692021    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 03 14:44:56.114159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:56.114132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:56.690912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:56.690871    2561 patch_prober.go:28] interesting pod/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 03 14:44:56.691425 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:56.690939    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 03 14:44:57.691173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:57.691139    2561 patch_prober.go:28] interesting pod/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 03 14:44:57.691565 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:57.691217    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 03 14:44:58.691360 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:58.691329    2561 patch_prober.go:28] interesting pod/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 03 14:44:58.691751 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:58.691380    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 03 14:44:59.695012 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:59.694980    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:44:59.695979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:44:59.695959    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:45:01.502132 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:01.502100    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:45:20.900481 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:20.900438    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:45:34.197715 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:34.197684    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:45:46.092199 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:46.092157    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:45:51.663560 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:51.663526    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:45:51.664271 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:51.663726    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerName="maas-api" containerID="cri-o://d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba" gracePeriod=30
Jul 03 14:45:52.111146 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.111125    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:45:52.179646 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.179619    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls\") pod \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") "
Jul 03 14:45:52.179777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.179714    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2bd6\" (UniqueName: \"kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6\") pod \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") "
Jul 03 14:45:52.179777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.179743    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca\") pod \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\" (UID: \"fbda208f-2e8e-43c7-9674-14c8e8fc6612\") "
Jul 03 14:45:52.180137 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.180110    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbda208f-2e8e-43c7-9674-14c8e8fc6612" (UID: "fbda208f-2e8e-43c7-9674-14c8e8fc6612"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:45:52.181794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.181767    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbda208f-2e8e-43c7-9674-14c8e8fc6612" (UID: "fbda208f-2e8e-43c7-9674-14c8e8fc6612"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:45:52.181896 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.181847    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6" (OuterVolumeSpecName: "kube-api-access-x2bd6") pod "fbda208f-2e8e-43c7-9674-14c8e8fc6612" (UID: "fbda208f-2e8e-43c7-9674-14c8e8fc6612"). InnerVolumeSpecName "kube-api-access-x2bd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:45:52.280940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.280909    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbda208f-2e8e-43c7-9674-14c8e8fc6612-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:45:52.280940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.280936    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2bd6\" (UniqueName: \"kubernetes.io/projected/fbda208f-2e8e-43c7-9674-14c8e8fc6612-kube-api-access-x2bd6\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:45:52.281089 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.280951    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbda208f-2e8e-43c7-9674-14c8e8fc6612-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:45:52.486428 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.486345    2561 generic.go:356] "Generic (PLEG): container finished" podID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerID="d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba" exitCode=0
Jul 03 14:45:52.486428 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.486388    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"
Jul 03 14:45:52.486653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.486440    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" event={"ID":"fbda208f-2e8e-43c7-9674-14c8e8fc6612","Type":"ContainerDied","Data":"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"}
Jul 03 14:45:52.486653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.486498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" event={"ID":"fbda208f-2e8e-43c7-9674-14c8e8fc6612","Type":"ContainerDied","Data":"002f0f8f6fecd84cf98b8d36f725da993b9d79beb14d0b394eba75750b22cd85"}
Jul 03 14:45:52.486653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.486528    2561 scope.go:117] "RemoveContainer" containerID="d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"
Jul 03 14:45:52.495151 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.495137    2561 scope.go:117] "RemoveContainer" containerID="d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"
Jul 03 14:45:52.495411 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:45:52.495393    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba\": container with ID starting with d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba not found: ID does not exist" containerID="d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"
Jul 03 14:45:52.495467 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.495421    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba"} err="failed to get container status \"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba\": rpc error: code = NotFound desc = could not find container \"d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba\": container with ID starting with d4008e782fc039606f84a8148fcb3131d010578e36f50944aa68a1a041fb2dba not found: ID does not exist"
Jul 03 14:45:52.510101 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.510081    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:45:52.517268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:52.517244    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm"]
Jul 03 14:45:54.191901 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:54.191872    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" path="/var/lib/kubelet/pods/fbda208f-2e8e-43c7-9674-14c8e8fc6612/volumes"
Jul 03 14:45:55.085941 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:45:55.085902    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-c1cb72-67b4496b5c-xkdfm" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.42:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 03 14:46:00.831559 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:00.831481    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:46:33.951107 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:33.951072    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:46:33.951633 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:33.951300    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy" containerID="cri-o://f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e" gracePeriod=30
Jul 03 14:46:35.136487 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.136455    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:46:35.136976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.136699    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" podUID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" containerName="maas-api" containerID="cri-o://87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19" gracePeriod=30
Jul 03 14:46:35.382056 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.382033    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:46:35.446764 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.446691    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") pod \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") "
Jul 03 14:46:35.446764 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.446733    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca\") pod \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") "
Jul 03 14:46:35.446994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.446822    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxvr7\" (UniqueName: \"kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7\") pod \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\" (UID: \"13bbdb98-746a-4162-bbf3-651b2b4c1abe\") "
Jul 03 14:46:35.447154 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.447130    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13bbdb98-746a-4162-bbf3-651b2b4c1abe" (UID: "13bbdb98-746a-4162-bbf3-651b2b4c1abe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:46:35.448828 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.448809    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13bbdb98-746a-4162-bbf3-651b2b4c1abe" (UID: "13bbdb98-746a-4162-bbf3-651b2b4c1abe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:46:35.448930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.448916    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7" (OuterVolumeSpecName: "kube-api-access-wxvr7") pod "13bbdb98-746a-4162-bbf3-651b2b4c1abe" (UID: "13bbdb98-746a-4162-bbf3-651b2b4c1abe"). InnerVolumeSpecName "kube-api-access-wxvr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:46:35.547579 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.547542    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13bbdb98-746a-4162-bbf3-651b2b4c1abe-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:35.547579 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.547574    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bbdb98-746a-4162-bbf3-651b2b4c1abe-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:35.547579 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.547584    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxvr7\" (UniqueName: \"kubernetes.io/projected/13bbdb98-746a-4162-bbf3-651b2b4c1abe-kube-api-access-wxvr7\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:35.692439 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.692406    2561 generic.go:356] "Generic (PLEG): container finished" podID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" containerID="87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19" exitCode=0
Jul 03 14:46:35.692592 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.692455    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"
Jul 03 14:46:35.692592 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.692489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" event={"ID":"13bbdb98-746a-4162-bbf3-651b2b4c1abe","Type":"ContainerDied","Data":"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"}
Jul 03 14:46:35.692592 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.692542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw" event={"ID":"13bbdb98-746a-4162-bbf3-651b2b4c1abe","Type":"ContainerDied","Data":"0db189ba626419df82629f3853bf9475b992e85eeb783afb878396c0b66abe58"}
Jul 03 14:46:35.692592 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.692563    2561 scope.go:117] "RemoveContainer" containerID="87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"
Jul 03 14:46:35.701503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.701482    2561 scope.go:117] "RemoveContainer" containerID="87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"
Jul 03 14:46:35.701762 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:46:35.701746    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19\": container with ID starting with 87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19 not found: ID does not exist" containerID="87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"
Jul 03 14:46:35.701813 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.701771    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19"} err="failed to get container status \"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19\": rpc error: code = NotFound desc = could not find container \"87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19\": container with ID starting with 87c62dea8f7d7fe3dc028b53702724d3e32d8d4895030f390e82cbda2ca16c19 not found: ID does not exist"
Jul 03 14:46:35.714203 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.714168    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:46:35.718585 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:35.718566    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e58844-869858cf74-vnzsw"]
Jul 03 14:46:36.190789 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:36.190749    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" path="/var/lib/kubelet/pods/13bbdb98-746a-4162-bbf3-651b2b4c1abe/volumes"
Jul 03 14:46:38.013644 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:38.013613    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:46:38.200273 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:38.200240    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:46:38.205323 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:38.205297    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:46:39.200508 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.200486    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:46:39.280208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280099    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280145    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280166    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280220    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhpmh\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280241    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280270    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280348    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280392    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280415    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token\") pod \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\" (UID: \"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81\") "
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280422    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:39.280503 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280461    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:39.280920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280722    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:46:39.280920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280834    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.280920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280853    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.280920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280868    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.281316 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.280967    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:39.281316 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.281218    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data" (OuterVolumeSpecName: "istio-data") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:39.282711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.282690    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:46:39.282839 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.282772    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token" (OuterVolumeSpecName: "istio-token") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:46:39.282995 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.282977    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:39.283104 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.283089    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh" (OuterVolumeSpecName: "kube-api-access-jhpmh") pod "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" (UID: "fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81"). InnerVolumeSpecName "kube-api-access-jhpmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:46:39.381759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381727    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.381759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381754    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.381759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381763    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.381992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381770    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.381992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381778    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhpmh\" (UniqueName: \"kubernetes.io/projected/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-kube-api-access-jhpmh\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.381992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.381787    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:39.713938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.713901    2561 generic.go:356] "Generic (PLEG): container finished" podID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerID="f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e" exitCode=0
Jul 03 14:46:39.714129 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.713956    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"
Jul 03 14:46:39.714129 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.713955    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerDied","Data":"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"}
Jul 03 14:46:39.714129 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.714083    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx" event={"ID":"fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81","Type":"ContainerDied","Data":"ff4f3da9bc2f33b302171397779fb4fef292f2c707146b54761afeb8268abd9d"}
Jul 03 14:46:39.714129 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.714117    2561 scope.go:117] "RemoveContainer" containerID="f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"
Jul 03 14:46:39.723254 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.723238    2561 scope.go:117] "RemoveContainer" containerID="20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"
Jul 03 14:46:39.731411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.731392    2561 scope.go:117] "RemoveContainer" containerID="f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"
Jul 03 14:46:39.731655 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:46:39.731637    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e\": container with ID starting with f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e not found: ID does not exist" containerID="f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"
Jul 03 14:46:39.731700 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.731665    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e"} err="failed to get container status \"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e\": rpc error: code = NotFound desc = could not find container \"f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e\": container with ID starting with f5d802b167e6011b021b5b4a5f0abd5f50fa5f92ec15782661e1f3f562c11b6e not found: ID does not exist"
Jul 03 14:46:39.731700 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.731682    2561 scope.go:117] "RemoveContainer" containerID="20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"
Jul 03 14:46:39.731924 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:46:39.731908    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531\": container with ID starting with 20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531 not found: ID does not exist" containerID="20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"
Jul 03 14:46:39.731961 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.731929    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531"} err="failed to get container status \"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531\": rpc error: code = NotFound desc = could not find container \"20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531\": container with ID starting with 20fde36f66b33a0ec4d34a3b67c3e3316e56722d669d3aebc0d539205c2ac531 not found: ID does not exist"
Jul 03 14:46:39.737460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.737440    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:46:39.742659 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:39.742639    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c1cb72-openshift-default-7b9b7d74b9-n9xbx"]
Jul 03 14:46:40.197654 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:40.197618    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" path="/var/lib/kubelet/pods/fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81/volumes"
Jul 03 14:46:41.277405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:41.277373    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:46:41.277883 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:41.277615    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" containerID="cri-o://03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70" gracePeriod=30
Jul 03 14:46:42.159457 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159428    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:46:42.159768 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159756    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerName="maas-api"
Jul 03 14:46:42.159824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerName="maas-api"
Jul 03 14:46:42.159824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159779    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:42.159824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159784    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:42.159824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159803    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" containerName="maas-api"
Jul 03 14:46:42.159824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159810    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" containerName="maas-api"
Jul 03 14:46:42.159996 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159865    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbda208f-2e8e-43c7-9674-14c8e8fc6612" containerName="maas-api"
Jul 03 14:46:42.159996 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159873    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:42.159996 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.159879    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="13bbdb98-746a-4162-bbf3-651b2b4c1abe" containerName="maas-api"
Jul 03 14:46:42.162892 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.162875    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.166025 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.166008    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-523e2d-openshift-default-dockercfg-qf7sk\""
Jul 03 14:46:42.178461 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.178437    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:46:42.315932 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.315891    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.315955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5kv8\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316233    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316266    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316300    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.316460 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.316352    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.416971    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417025    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417047    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417068    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417094    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417242    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5kv8\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417396 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417665 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417444    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417665 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417484    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417665 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417620    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417782 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417737    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.417968 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.417951    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.419596 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.419569    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.419782 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.419760    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.426434 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.426411    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.426789 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.426768    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5kv8\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8\") pod \"e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") " pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.473141 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.473114    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:42.612949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.612924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:46:42.615267 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:46:42.615239    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec043381_052c_4c8e_ba80_64457f7de948.slice/crio-6bdd85f279864f7654bb492183e7ec282cb554730ab8d4f6815d0a36fad334ad WatchSource:0}: Error finding container 6bdd85f279864f7654bb492183e7ec282cb554730ab8d4f6815d0a36fad334ad: Status 404 returned error can't find the container with id 6bdd85f279864f7654bb492183e7ec282cb554730ab8d4f6815d0a36fad334ad
Jul 03 14:46:42.617376 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.617347    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:42.617447 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.617406    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:42.617447 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.617440    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:42.731791 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.731753    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerStarted","Data":"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"}
Jul 03 14:46:42.731978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.731808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerStarted","Data":"6bdd85f279864f7654bb492183e7ec282cb554730ab8d4f6815d0a36fad334ad"}
Jul 03 14:46:42.770165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:42.770102    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podStartSLOduration=0.770082187 podStartE2EDuration="770.082187ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:46:42.764692242 +0000 UTC m=+2105.178588694" watchObservedRunningTime="2026-07-03 14:46:42.770082187 +0000 UTC m=+2105.183978637"
Jul 03 14:46:43.473509 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:43.473473    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:43.478841 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:43.478811    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:43.737128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:43.737045    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:43.738071 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:43.738052    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:44.698409 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:44.698374    2561 patch_prober.go:28] interesting pod/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 03 14:46:44.698772 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:44.698433    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 03 14:46:45.081992 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.081909    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:46:45.082291 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.082278    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:45.082337 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.082293    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:45.082389 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.082365    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea4fcf9-a4ca-4fd0-aff5-0d7658ae6d81" containerName="istio-proxy"
Jul 03 14:46:45.086103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.086073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.089826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.089803    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-523e2d-serving-cert\""
Jul 03 14:46:45.090029 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.089856    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:46:45.103624 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.103601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:46:45.243576 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.243539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.243767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.243605    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.243767 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.243667    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqqds\" (UniqueName: \"kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.345032 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.345002    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.345230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.345066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqqds\" (UniqueName: \"kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.345230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.345107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.345764 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.345745    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.347605 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.347586    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.354413 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.354391    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqqds\" (UniqueName: \"kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds\") pod \"maas-api-e2e-api-a-523e2d-db66c6449-mfnrx\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") " pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.397823 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.397793    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:45.534128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.534095    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:46:45.537442 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:46:45.537416    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68a08c0_e4f1_4a6d_8c16_7d28077e65e1.slice/crio-f329916372f308a61b25ecb8ad03a38099ddff4a8e0c250d6d9912f90a7be464 WatchSource:0}: Error finding container f329916372f308a61b25ecb8ad03a38099ddff4a8e0c250d6d9912f90a7be464: Status 404 returned error can't find the container with id f329916372f308a61b25ecb8ad03a38099ddff4a8e0c250d6d9912f90a7be464
Jul 03 14:46:45.750416 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:45.750330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" event={"ID":"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1","Type":"ContainerStarted","Data":"f329916372f308a61b25ecb8ad03a38099ddff4a8e0c250d6d9912f90a7be464"}
Jul 03 14:46:46.523563 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.523539    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:46:46.655409 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655422    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655447    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655477    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655542    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brk9f\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655576    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655608 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655599    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655615    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655650    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy\") pod \"1597e6ca-7729-4741-ada8-3d464bb91e47\" (UID: \"1597e6ca-7729-4741-ada8-3d464bb91e47\") "
Jul 03 14:46:46.655912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655727    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:46.655912 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655885    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:46.656118 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.655996    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.656118 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.656016    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.656118 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.656056    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:46.656284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.656202    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:46:46.656754 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.656728    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data" (OuterVolumeSpecName: "istio-data") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:46.657945 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.657922    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:46:46.658088 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.657943    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:46:46.658349 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.658331    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f" (OuterVolumeSpecName: "kube-api-access-brk9f") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "kube-api-access-brk9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:46:46.658406 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.658353    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token" (OuterVolumeSpecName: "istio-token") pod "1597e6ca-7729-4741-ada8-3d464bb91e47" (UID: "1597e6ca-7729-4741-ada8-3d464bb91e47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:46:46.756651 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756616    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brk9f\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-kube-api-access-brk9f\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.756651 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756651    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756666    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1597e6ca-7729-4741-ada8-3d464bb91e47-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756677    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756685    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756693    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1597e6ca-7729-4741-ada8-3d464bb91e47-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.756701    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1597e6ca-7729-4741-ada8-3d464bb91e47-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:46:46.757821 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.757794    2561 generic.go:356] "Generic (PLEG): container finished" podID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerID="03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70" exitCode=0
Jul 03 14:46:46.757938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.757842    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"
Jul 03 14:46:46.757938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.757876    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerDied","Data":"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"}
Jul 03 14:46:46.757938 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.757925    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr" event={"ID":"1597e6ca-7729-4741-ada8-3d464bb91e47","Type":"ContainerDied","Data":"d4168beafa23a8952b6d43d720b169aff86c75ad8c9c683a2b154c1e89a3d520"}
Jul 03 14:46:46.758050 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.757944    2561 scope.go:117] "RemoveContainer" containerID="03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"
Jul 03 14:46:46.759663 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.759634    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" event={"ID":"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1","Type":"ContainerStarted","Data":"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"}
Jul 03 14:46:46.759799 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.759737    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:46.767741 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.767727    2561 scope.go:117] "RemoveContainer" containerID="db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"
Jul 03 14:46:46.775432 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.775417    2561 scope.go:117] "RemoveContainer" containerID="03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"
Jul 03 14:46:46.775702 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:46:46.775682    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70\": container with ID starting with 03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70 not found: ID does not exist" containerID="03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"
Jul 03 14:46:46.775746 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.775710    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70"} err="failed to get container status \"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70\": rpc error: code = NotFound desc = could not find container \"03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70\": container with ID starting with 03a85b45837c6be8f12459408e3ccac729dde2c9a610887c73c2fe4411486e70 not found: ID does not exist"
Jul 03 14:46:46.775746 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.775726    2561 scope.go:117] "RemoveContainer" containerID="db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"
Jul 03 14:46:46.775969 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:46:46.775950    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be\": container with ID starting with db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be not found: ID does not exist" containerID="db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"
Jul 03 14:46:46.776017 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.775974    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be"} err="failed to get container status \"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be\": rpc error: code = NotFound desc = could not find container \"db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be\": container with ID starting with db5631b20a8e2951b4fc86efa334fd984a36bc8effb5d52d465025b838e835be not found: ID does not exist"
Jul 03 14:46:46.781829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.781792    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" podStartSLOduration=1.312847454 podStartE2EDuration="1.781778858s" totalImagesPullingTime="468.931404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:46:46.779431768 +0000 UTC m=+2109.193328218" watchObservedRunningTime="2026-07-03 14:46:46.781778858 +0000 UTC m=+2109.195675307"
Jul 03 14:46:46.795114 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.795091    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:46:46.799078 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:46.799056    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e58844-openshift-default-7bd4d9c55-gqlsr"]
Jul 03 14:46:48.191017 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:48.190983    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" path="/var/lib/kubelet/pods/1597e6ca-7729-4741-ada8-3d464bb91e47/volumes"
Jul 03 14:46:48.315960 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:48.315928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:46:50.144219 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.143625    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:46:50.150221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.144170    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:46:50.150221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.148247    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:46:50.150221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.148312    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:46:50.150221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.148323    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:46:50.150221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.148450    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:46:50.175208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.173653    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:46:50.175208 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.173783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.176619 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.176598    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e921af-openshift-default-dockercfg-k58ht\""
Jul 03 14:46:50.288985 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.288948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.288995    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289018    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289159 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.289366 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.289298    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp7t9\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.390783 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.390745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.390942 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.390845    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp7t9\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.390942 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.390905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.390958    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.390989    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391045 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391016    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391221 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391144    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391395 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391364    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391472 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391443    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391541    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391719 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391695    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.391780 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.391678    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.393223 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.393172    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.393805 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.393783    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.400529 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.400471    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.400711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.400691    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp7t9\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9\") pod \"e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") " pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.486102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.486067    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:50.690847 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.690822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:46:50.693217 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:46:50.693143    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738acf7c_b005_482e_90c1_5af5406dd5a0.slice/crio-33edc85b9409233f4b207d478f114d1cd906f75305cf8120e30ab83c9c42015f WatchSource:0}: Error finding container 33edc85b9409233f4b207d478f114d1cd906f75305cf8120e30ab83c9c42015f: Status 404 returned error can't find the container with id 33edc85b9409233f4b207d478f114d1cd906f75305cf8120e30ab83c9c42015f
Jul 03 14:46:50.695587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.695546    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:50.695686 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.695634    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:50.695686 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.695676    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:50.782824 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:50.782790    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerStarted","Data":"33edc85b9409233f4b207d478f114d1cd906f75305cf8120e30ab83c9c42015f"}
Jul 03 14:46:51.789809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:51.789774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerStarted","Data":"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"}
Jul 03 14:46:51.813276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:51.813226    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podStartSLOduration=1.813209396 podStartE2EDuration="1.813209396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:46:51.811238918 +0000 UTC m=+2114.225135367" watchObservedRunningTime="2026-07-03 14:46:51.813209396 +0000 UTC m=+2114.227105836"
Jul 03 14:46:52.486921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:52.486884    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:52.491887 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:52.491861    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:52.772629 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:52.772550    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:46:52.795421 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:52.795397    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:52.796434 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:52.796412    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:46:56.818105 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.818072    2561 generic.go:356] "Generic (PLEG): container finished" podID="ec043381-052c-4c8e-ba80-64457f7de948" containerID="f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967" exitCode=0
Jul 03 14:46:56.818575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.818147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerDied","Data":"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"}
Jul 03 14:46:56.818575 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.818547    2561 scope.go:117] "RemoveContainer" containerID="f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"
Jul 03 14:46:56.819667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.819642    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:56.819725 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.819710    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:56.819760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:56.819747    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:46:57.825058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:57.825025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerStarted","Data":"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"}
Jul 03 14:46:58.474224 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:58.474175    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:58.475139 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:58.475116    2561 patch_prober.go:28] interesting pod/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 03 14:46:58.475255 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:58.475191    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 03 14:46:58.738023 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:58.737942    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:46:59.217572 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:59.217527    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:46:59.474237 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:59.474130    2561 patch_prober.go:28] interesting pod/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 03 14:46:59.474237 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:46:59.474223    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 03 14:47:00.474479 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:00.474436    2561 patch_prober.go:28] interesting pod/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 03 14:47:00.474986 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:00.474504    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 03 14:47:01.474806 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:01.474316    2561 patch_prober.go:28] interesting pod/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 03 14:47:01.474806 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:01.474399    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 03 14:47:02.477587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:02.477557    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:47:02.478639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:02.478618    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:47:07.477143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.477108    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:47:07.477604 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.477337    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" podUID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" containerName="maas-api" containerID="cri-o://c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401" gracePeriod=30
Jul 03 14:47:07.747097 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.747077    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:47:07.797149 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.797115    2561 patch_prober.go:28] interesting pod/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 03 14:47:07.797303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.797210    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 03 14:47:07.856682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.856621    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqqds\" (UniqueName: \"kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds\") pod \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") "
Jul 03 14:47:07.856682 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.856662    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca\") pod \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") "
Jul 03 14:47:07.856842 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.856717    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls\") pod \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\" (UID: \"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1\") "
Jul 03 14:47:07.857043 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.857020    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" (UID: "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:47:07.858829 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.858800    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds" (OuterVolumeSpecName: "kube-api-access-gqqds") pod "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" (UID: "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1"). InnerVolumeSpecName "kube-api-access-gqqds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:07.858917 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.858897    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" (UID: "e68a08c0-e4f1-4a6d-8c16-7d28077e65e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:47:07.880707 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.880685    2561 generic.go:356] "Generic (PLEG): container finished" podID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerID="48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c" exitCode=0
Jul 03 14:47:07.880809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.880762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerDied","Data":"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"}
Jul 03 14:47:07.881220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.881175    2561 scope.go:117] "RemoveContainer" containerID="48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"
Jul 03 14:47:07.882012 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.881991    2561 generic.go:356] "Generic (PLEG): container finished" podID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" containerID="c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401" exitCode=0
Jul 03 14:47:07.882103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882082    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" event={"ID":"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1","Type":"ContainerDied","Data":"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"}
Jul 03 14:47:07.882173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882126    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"
Jul 03 14:47:07.882173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882140    2561 scope.go:117] "RemoveContainer" containerID="c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"
Jul 03 14:47:07.882314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882127    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx" event={"ID":"e68a08c0-e4f1-4a6d-8c16-7d28077e65e1","Type":"ContainerDied","Data":"f329916372f308a61b25ecb8ad03a38099ddff4a8e0c250d6d9912f90a7be464"}
Jul 03 14:47:07.882314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882229    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:07.882314 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882299    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:07.882441 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.882348    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:07.893259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.893198    2561 scope.go:117] "RemoveContainer" containerID="c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"
Jul 03 14:47:07.893479 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:07.893460    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401\": container with ID starting with c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401 not found: ID does not exist" containerID="c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"
Jul 03 14:47:07.893557 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.893489    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401"} err="failed to get container status \"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401\": rpc error: code = NotFound desc = could not find container \"c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401\": container with ID starting with c26c90fd9ffb99707f6455213b45811a5f5d33983325a5b8a00a0c2d28e4a401 not found: ID does not exist"
Jul 03 14:47:07.932558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.932535    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:47:07.939826 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.939753    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-523e2d-db66c6449-mfnrx"]
Jul 03 14:47:07.957805 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.957776    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:07.957805 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.957809    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqqds\" (UniqueName: \"kubernetes.io/projected/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-kube-api-access-gqqds\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:07.958011 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:07.957824    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:08.192482 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:08.192401    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" path="/var/lib/kubelet/pods/e68a08c0-e4f1-4a6d-8c16-7d28077e65e1/volumes"
Jul 03 14:47:08.889240 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:08.889196    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerStarted","Data":"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"}
Jul 03 14:47:09.486675 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:09.486632    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:09.486894 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:09.486870    2561 patch_prober.go:28] interesting pod/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 03 14:47:09.486955 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:09.486922    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 03 14:47:10.486889 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:10.486859    2561 patch_prober.go:28] interesting pod/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 03 14:47:10.487329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:10.486912    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 03 14:47:11.487240 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:11.487175    2561 patch_prober.go:28] interesting pod/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 03 14:47:11.487638 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:11.487257    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 03 14:47:12.486914 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:12.486871    2561 patch_prober.go:28] interesting pod/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 03 14:47:12.487122 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:12.486934    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 03 14:47:12.613132 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:12.613094    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:47:13.490901 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:13.490870    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:13.491143 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:13.491122    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:13.491905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:13.491885    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:13.607306 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:13.607268    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:47:13.607563 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:13.607538    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" containerID="cri-o://0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a" gracePeriod=30
Jul 03 14:47:17.480739 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:17.480701    2561 patch_prober.go:28] interesting pod/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 03 14:47:17.481128 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:17.480762    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 03 14:47:18.861679 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.861653    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:47:18.942264 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.942160    2561 generic.go:356] "Generic (PLEG): container finished" podID="ec043381-052c-4c8e-ba80-64457f7de948" containerID="0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a" exitCode=0
Jul 03 14:47:18.942264 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.942230    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"
Jul 03 14:47:18.942496 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.942256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerDied","Data":"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"}
Jul 03 14:47:18.942496 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.942320    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr" event={"ID":"ec043381-052c-4c8e-ba80-64457f7de948","Type":"ContainerDied","Data":"6bdd85f279864f7654bb492183e7ec282cb554730ab8d4f6815d0a36fad334ad"}
Jul 03 14:47:18.942496 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.942354    2561 scope.go:117] "RemoveContainer" containerID="0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"
Jul 03 14:47:18.952152 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.952135    2561 scope.go:117] "RemoveContainer" containerID="f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"
Jul 03 14:47:18.959920 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.959906    2561 scope.go:117] "RemoveContainer" containerID="0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"
Jul 03 14:47:18.960170 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:18.960152    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a\": container with ID starting with 0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a not found: ID does not exist" containerID="0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"
Jul 03 14:47:18.960264 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.960199    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a"} err="failed to get container status \"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a\": rpc error: code = NotFound desc = could not find container \"0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a\": container with ID starting with 0c640f90735cc31a55545ad7904ca99d3e81e5a6b7a5f4cd0c9c63c53484c54a not found: ID does not exist"
Jul 03 14:47:18.960264 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.960219    2561 scope.go:117] "RemoveContainer" containerID="f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"
Jul 03 14:47:18.960455 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:18.960441    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967\": container with ID starting with f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967 not found: ID does not exist" containerID="f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"
Jul 03 14:47:18.960500 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.960460    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967"} err="failed to get container status \"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967\": rpc error: code = NotFound desc = could not find container \"f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967\": container with ID starting with f42acd9856c0a3d489a3fddbbe199b5cbe5cddb64d16fae1137d4b2d15bd9967 not found: ID does not exist"
Jul 03 14:47:18.966787 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966770    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5kv8\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.966860 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966806    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.966860 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966845    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.966967 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966879    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.966967 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966895    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.966967 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.966920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.967116 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967057    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.967168 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967119    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.967168 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data\") pod \"ec043381-052c-4c8e-ba80-64457f7de948\" (UID: \"ec043381-052c-4c8e-ba80-64457f7de948\") "
Jul 03 14:47:18.967168 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967162    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.967369 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967200    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.967426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967298    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:47:18.967426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967372    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.967726 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967621    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec043381-052c-4c8e-ba80-64457f7de948-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.967786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967729    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.967786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967742    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.967786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.967756    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.968070 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.968049    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.969216 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.969173    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:18.969305 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.969284    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8" (OuterVolumeSpecName: "kube-api-access-q5kv8") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "kube-api-access-q5kv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:18.969725 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.969696    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:47:18.969822 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:18.969701    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec043381-052c-4c8e-ba80-64457f7de948" (UID: "ec043381-052c-4c8e-ba80-64457f7de948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:19.069068 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.069031    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec043381-052c-4c8e-ba80-64457f7de948-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:19.069068 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.069060    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:19.069068 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.069069    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5kv8\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-kube-api-access-q5kv8\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:19.069068 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.069078    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec043381-052c-4c8e-ba80-64457f7de948-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:19.069375 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.069088    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec043381-052c-4c8e-ba80-64457f7de948-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:19.265667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.265631    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:47:19.272702 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:19.272674    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-523e2d-openshift-default-5c459cbf7c-7bhkr"]
Jul 03 14:47:20.191120 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:20.191086    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec043381-052c-4c8e-ba80-64457f7de948" path="/var/lib/kubelet/pods/ec043381-052c-4c8e-ba80-64457f7de948/volumes"
Jul 03 14:47:21.053384 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.053348    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:47:21.053581 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.053562    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy" containerID="cri-o://08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f" gracePeriod=30
Jul 03 14:47:21.497152 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.497115    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:47:21.989880 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.989848    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:47:21.990211 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990197    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" containerName="maas-api"
Jul 03 14:47:21.990267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990213    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" containerName="maas-api"
Jul 03 14:47:21.990267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990234    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.990267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990239    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.990267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990248    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.990267 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990253    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.990415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990313    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.990415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990323    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1597e6ca-7729-4741-ada8-3d464bb91e47" containerName="istio-proxy"
Jul 03 14:47:21.990415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990329    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68a08c0-e4f1-4a6d-8c16-7d28077e65e1" containerName="maas-api"
Jul 03 14:47:21.990415 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.990335    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec043381-052c-4c8e-ba80-64457f7de948" containerName="istio-proxy"
Jul 03 14:47:21.994451 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.994433    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:21.996792 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:21.996773    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6ebb6a-openshift-default-dockercfg-n4vnm\""
Jul 03 14:47:22.005604 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.005582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:47:22.097098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097098 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxwv\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097238    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097260    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.097534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.097424    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198374 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198379    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198490    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxwv\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198542    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198816    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.198990 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.198927    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.199049 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.199007    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.199102 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.199053    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.199324 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.199301    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.200969 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.200948    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.201063 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.201020    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.207236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.207214    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.207470 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.207450    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxwv\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv\") pod \"e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") " pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.304691 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.304606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:22.443598 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.443575    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:47:22.445511 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:47:22.445485    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277e3aef_4df5_49b5_a469_1e8912a8f443.slice/crio-957442d2369a992ece70512afe2f846c26b8bb586997089a55ad30e935d328df WatchSource:0}: Error finding container 957442d2369a992ece70512afe2f846c26b8bb586997089a55ad30e935d328df: Status 404 returned error can't find the container with id 957442d2369a992ece70512afe2f846c26b8bb586997089a55ad30e935d328df
Jul 03 14:47:22.447751 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.447719    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:22.447836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.447783    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:22.447836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.447821    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:22.963846 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.963807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerStarted","Data":"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"}
Jul 03 14:47:22.964254 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.963863    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerStarted","Data":"957442d2369a992ece70512afe2f846c26b8bb586997089a55ad30e935d328df"}
Jul 03 14:47:22.993229 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:22.993161    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podStartSLOduration=1.993146302 podStartE2EDuration="1.993146302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:22.990574385 +0000 UTC m=+2145.404470834" watchObservedRunningTime="2026-07-03 14:47:22.993146302 +0000 UTC m=+2145.407042751"
Jul 03 14:47:23.305021 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:23.304944    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:23.309854 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:23.309833    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:23.968531 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:23.968501    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:23.969520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:23.969500    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:25.241227 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.241164    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:47:25.244749 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.244725    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.247411 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.247378    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-6ebb6a-serving-cert\""
Jul 03 14:47:25.247558 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.247478    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4fqb\""
Jul 03 14:47:25.257050 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.257029    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:47:25.328455 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.328414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.328627 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.328468    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.328627 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.328564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh74x\" (UniqueName: \"kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.429622 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.429592    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.429817 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.429632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.429817 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.429661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh74x\" (UniqueName: \"kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.429817 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:25.429734    2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-6ebb6a-serving-cert: secret "maas-api-e2e-shared-a-6ebb6a-serving-cert" not found
Jul 03 14:47:25.429817 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:25.429796    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls podName:36dfd345-d7a3-43fc-8be2-926634f9bc96 nodeName:}" failed. No retries permitted until 2026-07-03 14:47:25.929779705 +0000 UTC m=+2148.343676132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls") pod "maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" (UID: "36dfd345-d7a3-43fc-8be2-926634f9bc96") : secret "maas-api-e2e-shared-a-6ebb6a-serving-cert" not found
Jul 03 14:47:25.430271 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.430252    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.439122 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.439091    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh74x\" (UniqueName: \"kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.934836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.934805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:25.938864 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:25.938830    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") pod \"maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") " pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:26.155799 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.155770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:26.308116 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.308081    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:47:26.309168 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.309014    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:26.311103 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:47:26.311072    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36dfd345_d7a3_43fc_8be2_926634f9bc96.slice/crio-3f3c698999f6455800e1894560ae46753d5778860a020e21defb3d5f07ca2f45 WatchSource:0}: Error finding container 3f3c698999f6455800e1894560ae46753d5778860a020e21defb3d5f07ca2f45: Status 404 returned error can't find the container with id 3f3c698999f6455800e1894560ae46753d5778860a020e21defb3d5f07ca2f45
Jul 03 14:47:26.440905 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.440871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.440911    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.440938    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.440969    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441020    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp7t9\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441087 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441050    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441133    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441223    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441252    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data\") pod \"738acf7c-b005-482e-90c1-5af5406dd5a0\" (UID: \"738acf7c-b005-482e-90c1-5af5406dd5a0\") "
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441328    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441349    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:47:26.441386 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441346    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:26.441711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441647    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/738acf7c-b005-482e-90c1-5af5406dd5a0-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.441711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441670    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.441711 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441686    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.441857 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.441777    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:26.442161 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.442139    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:26.443504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.443484    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:47:26.443797 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.443773    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:26.443797 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.443780    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9" (OuterVolumeSpecName: "kube-api-access-wp7t9") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "kube-api-access-wp7t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:26.443949 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.443874    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "738acf7c-b005-482e-90c1-5af5406dd5a0" (UID: "738acf7c-b005-482e-90c1-5af5406dd5a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543082    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543109    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543117    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543127    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp7t9\" (UniqueName: \"kubernetes.io/projected/738acf7c-b005-482e-90c1-5af5406dd5a0-kube-api-access-wp7t9\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543136    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/738acf7c-b005-482e-90c1-5af5406dd5a0-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.543157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.543144    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/738acf7c-b005-482e-90c1-5af5406dd5a0-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:47:26.983742 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.983706    2561 generic.go:356] "Generic (PLEG): container finished" podID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerID="08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f" exitCode=0
Jul 03 14:47:26.983931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.983760    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"
Jul 03 14:47:26.983931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.983786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerDied","Data":"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"}
Jul 03 14:47:26.983931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.983840    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz" event={"ID":"738acf7c-b005-482e-90c1-5af5406dd5a0","Type":"ContainerDied","Data":"33edc85b9409233f4b207d478f114d1cd906f75305cf8120e30ab83c9c42015f"}
Jul 03 14:47:26.983931 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.983874    2561 scope.go:117] "RemoveContainer" containerID="08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"
Jul 03 14:47:26.985529 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.985509    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" event={"ID":"36dfd345-d7a3-43fc-8be2-926634f9bc96","Type":"ContainerStarted","Data":"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"}
Jul 03 14:47:26.985667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.985556    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" event={"ID":"36dfd345-d7a3-43fc-8be2-926634f9bc96","Type":"ContainerStarted","Data":"3f3c698999f6455800e1894560ae46753d5778860a020e21defb3d5f07ca2f45"}
Jul 03 14:47:26.993349 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:26.993331    2561 scope.go:117] "RemoveContainer" containerID="48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"
Jul 03 14:47:27.001050 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.001034    2561 scope.go:117] "RemoveContainer" containerID="08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"
Jul 03 14:47:27.001305 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:27.001284    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f\": container with ID starting with 08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f not found: ID does not exist" containerID="08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"
Jul 03 14:47:27.001345 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.001315    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f"} err="failed to get container status \"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f\": rpc error: code = NotFound desc = could not find container \"08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f\": container with ID starting with 08c09721983c0c5dee096aa590169e648910658f6b9d9793eca402c0933d451f not found: ID does not exist"
Jul 03 14:47:27.001345 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.001332    2561 scope.go:117] "RemoveContainer" containerID="48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"
Jul 03 14:47:27.001577 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:47:27.001558    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c\": container with ID starting with 48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c not found: ID does not exist" containerID="48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"
Jul 03 14:47:27.001624 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.001583    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c"} err="failed to get container status \"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c\": rpc error: code = NotFound desc = could not find container \"48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c\": container with ID starting with 48db397ea6cf07906b6f1f24cdaf75b55d1b3938652e11f91f7b4da9946cb52c not found: ID does not exist"
Jul 03 14:47:27.006606 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.006568    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" podStartSLOduration=1.6809855310000001 podStartE2EDuration="2.00655692s" totalImagesPullingTime="325.571389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:27.004990219 +0000 UTC m=+2149.418886666" watchObservedRunningTime="2026-07-03 14:47:27.00655692 +0000 UTC m=+2149.420453411"
Jul 03 14:47:27.020165 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.019754    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:47:27.021662 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.021639    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e921af-openshift-default-6ffbd47f4f-lfmgz"]
Jul 03 14:47:27.991287 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:27.991260    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:28.191284 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:28.191245    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" path="/var/lib/kubelet/pods/738acf7c-b005-482e-90c1-5af5406dd5a0/volumes"
Jul 03 14:47:29.799743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:29.799668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:47:30.367884 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.367846    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:47:30.368336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.368317    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:30.368336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.368337    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:30.368488 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.368451    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:30.368488 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.368465    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:30.373100 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.373079    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.375439 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.375420    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1dd877-openshift-default-dockercfg-ntbh4\""
Jul 03 14:47:30.385941 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.385920    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:47:30.481095 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481058    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481095 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnln\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481195    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481372 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481302    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481372    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.481564 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.481433    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582329 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582338    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582526 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582703    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnln\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582761    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582805    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582842    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.582978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.582889    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.583340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.583323    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.585002 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.584977    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.585216 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.585173    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.590652 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.590628    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.590743 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.590680    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnln\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln\") pod \"e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") " pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.683689 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.683609    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:30.863883 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.853600    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:47:30.863883 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:47:30.858758    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfabebd2_27e2_41b3_991f_5a5a6a3253f1.slice/crio-56417980be8350714b822f90497f38eac3115998b91cc511846570d25b28a49c WatchSource:0}: Error finding container 56417980be8350714b822f90497f38eac3115998b91cc511846570d25b28a49c: Status 404 returned error can't find the container with id 56417980be8350714b822f90497f38eac3115998b91cc511846570d25b28a49c
Jul 03 14:47:30.863883 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.862341    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:30.872324 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.870396    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:30.872324 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:30.870472    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:31.010051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:31.009962    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerStarted","Data":"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"}
Jul 03 14:47:31.010051 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:31.010034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerStarted","Data":"56417980be8350714b822f90497f38eac3115998b91cc511846570d25b28a49c"}
Jul 03 14:47:31.035131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:31.035071    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podStartSLOduration=1.035050783 podStartE2EDuration="1.035050783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:31.032673703 +0000 UTC m=+2153.446570156" watchObservedRunningTime="2026-07-03 14:47:31.035050783 +0000 UTC m=+2153.448947234"
Jul 03 14:47:31.684808 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:31.684776    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:31.689720 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:31.689691    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:32.015566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:32.015486    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:32.016492 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:32.016473    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:33.320882 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.320848    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:47:33.321421 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.321401    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:33.321499 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.321424    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="738acf7c-b005-482e-90c1-5af5406dd5a0" containerName="istio-proxy"
Jul 03 14:47:33.324650 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.324630    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.327076 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.327054    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-1dd877-serving-cert\""
Jul 03 14:47:33.333759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.333736    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:47:33.511445 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.511416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjjtx\" (UniqueName: \"kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.511620 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.511485    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.511620 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.511567    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.612407 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.612373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.612581 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.612434    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.612581 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.612496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjjtx\" (UniqueName: \"kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.613080 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.613057    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.614927 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.614908    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.624040 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.624014    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjjtx\" (UniqueName: \"kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx\") pod \"maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") " pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.637873 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.637852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:33.814759 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:33.814646    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:47:33.816869 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:47:33.816835    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90be159_8f67_4197_81ee_dc41a5f8e941.slice/crio-04caa7ca74a7deb66ac8de1f3cb2d19d0b15fcf8343d32d2d75e1e67b5e8f6cd WatchSource:0}: Error finding container 04caa7ca74a7deb66ac8de1f3cb2d19d0b15fcf8343d32d2d75e1e67b5e8f6cd: Status 404 returned error can't find the container with id 04caa7ca74a7deb66ac8de1f3cb2d19d0b15fcf8343d32d2d75e1e67b5e8f6cd
Jul 03 14:47:34.001476 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:34.001396    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:47:34.032599 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:34.032299    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" event={"ID":"b90be159-8f67-4197-81ee-dc41a5f8e941","Type":"ContainerStarted","Data":"04caa7ca74a7deb66ac8de1f3cb2d19d0b15fcf8343d32d2d75e1e67b5e8f6cd"}
Jul 03 14:47:35.038685 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:35.038647    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" event={"ID":"b90be159-8f67-4197-81ee-dc41a5f8e941","Type":"ContainerStarted","Data":"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"}
Jul 03 14:47:35.039071 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:35.038707    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:35.061552 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:35.061504    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" podStartSLOduration=1.7363919220000001 podStartE2EDuration="2.061488993s" totalImagesPullingTime="325.097071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:35.05879279 +0000 UTC m=+2157.472689239" watchObservedRunningTime="2026-07-03 14:47:35.061488993 +0000 UTC m=+2157.475385443"
Jul 03 14:47:38.055501 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.055470    2561 generic.go:356] "Generic (PLEG): container finished" podID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerID="97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55" exitCode=0
Jul 03 14:47:38.055907 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.055519    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerDied","Data":"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"}
Jul 03 14:47:38.055907 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.055860    2561 scope.go:117] "RemoveContainer" containerID="97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"
Jul 03 14:47:38.056914 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.056884    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:38.056994 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.056968    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:38.057030 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.057019    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:38.808018 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.807985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:47:38.969442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:38.969395    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:39.065359 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:39.065255    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerStarted","Data":"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"}
Jul 03 14:47:39.305003 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:39.304967    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:39.306251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:39.306217    2561 patch_prober.go:28] interesting pod/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 03 14:47:39.306379 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:39.306286    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 03 14:47:40.305896 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:40.305856    2561 patch_prober.go:28] interesting pod/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 03 14:47:40.306331 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:40.305915    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 03 14:47:41.058464 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:41.058428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:47:41.305879 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:41.305849    2561 patch_prober.go:28] interesting pod/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 03 14:47:41.306058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:41.305902    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 03 14:47:42.305363 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:42.305326    2561 patch_prober.go:28] interesting pod/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 03 14:47:42.305565 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:42.305383    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 03 14:47:43.310539 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:43.310466    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:43.311013 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:43.310940    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:43.311778 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:43.311753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:47:47.017298 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.017263    2561 patch_prober.go:28] interesting pod/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 03 14:47:47.017726 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.017318    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 03 14:47:47.117618 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.117583    2561 generic.go:356] "Generic (PLEG): container finished" podID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerID="176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a" exitCode=0
Jul 03 14:47:47.117786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.117638    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerDied","Data":"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"}
Jul 03 14:47:47.118034 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.118021    2561 scope.go:117] "RemoveContainer" containerID="176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"
Jul 03 14:47:47.119060 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.119035    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:47.119123 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.119107    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:47.119173 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:47.119162    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 03 14:47:48.124352 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:48.124318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerStarted","Data":"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"}
Jul 03 14:47:48.684405 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:48.684377    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:48.684617 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:48.684594    2561 patch_prober.go:28] interesting pod/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 03 14:47:48.684671 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:48.684643    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 03 14:47:49.684391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:49.684359    2561 patch_prober.go:28] interesting pod/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 03 14:47:49.684781 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:49.684414    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 03 14:47:50.684673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:50.684636    2561 patch_prober.go:28] interesting pod/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 03 14:47:50.685062 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:50.684694    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 03 14:47:51.683953 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:51.683915    2561 patch_prober.go:28] interesting pod/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 03 14:47:51.684142 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:51.683967    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 03 14:47:52.688843 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:52.688814    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:52.689285 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:52.689095    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:52.689844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:52.689825    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:47:59.003131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:47:59.003099    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:48:32.600828 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:48:32.600793    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:48:59.811777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:48:59.811695    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:49:17.700866 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:49:17.700821    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:49:30.797050 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:49:30.797009    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:49:47.609008 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:49:47.608971    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:50:03.308741 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:03.308702    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:50:20.460570 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:20.460536    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:50:37.399241 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:37.399160    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:50:46.604976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.604946    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:50:46.608310 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.608294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.612009 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.611982    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6ebb6a\"/\"sub-test-model-6ebb6a-kserve-self-signed-certs\""
Jul 03 14:50:46.612009 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.612003    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6ebb6a\"/\"default-dockercfg-9x89m\""
Jul 03 14:50:46.612228 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.611993    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6ebb6a\"/\"openshift-service-ca.crt\""
Jul 03 14:50:46.612228 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.612009    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6ebb6a\"/\"kube-root-ca.crt\""
Jul 03 14:50:46.619504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.619482    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:50:46.725175 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725146    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.725332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.725332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725235    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.725332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.725332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725303    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ll7x\" (UniqueName: \"kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.725332 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.725330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826430 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ll7x\" (UniqueName: \"kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826567 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826438    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826567 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826680 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826739 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826793 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826768    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.826990 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.826969    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.827268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.827044    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.827268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.827069    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.828816 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.828797    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.828991 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.828974    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.840216 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.840171    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ll7x\" (UniqueName: \"kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x\") pod \"sub-test-model-6ebb6a-kserve-774874cc4c-98k84\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") " pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:46.917398 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:46.917306    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:47.064825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:47.064793    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:50:47.065979 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:50:47.065954    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5affb4a_04e8_42a7_a2c9_fb9f52e86d8b.slice/crio-0429c373898c518cd88051dbf4ac86c05a90a2933a0397e18a87b6b1152fd01b WatchSource:0}: Error finding container 0429c373898c518cd88051dbf4ac86c05a90a2933a0397e18a87b6b1152fd01b: Status 404 returned error can't find the container with id 0429c373898c518cd88051dbf4ac86c05a90a2933a0397e18a87b6b1152fd01b
Jul 03 14:50:47.967597 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:47.967560    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" event={"ID":"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b","Type":"ContainerStarted","Data":"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"}
Jul 03 14:50:47.968081 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:47.967608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" event={"ID":"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b","Type":"ContainerStarted","Data":"0429c373898c518cd88051dbf4ac86c05a90a2933a0397e18a87b6b1152fd01b"}
Jul 03 14:50:47.985784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:47.985735    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" podStartSLOduration=1.985720802 podStartE2EDuration="1.985720802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:50:47.98565856 +0000 UTC m=+2350.399555020" watchObservedRunningTime="2026-07-03 14:50:47.985720802 +0000 UTC m=+2350.399617251"
Jul 03 14:50:51.907310 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:51.907274    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:50:56.917856 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:56.917813    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:56.918262 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:56.918065    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:56.930554 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:56.930531    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:50:57.020924 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:50:57.020896    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:51:29.605276 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:51:29.605239    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:51:38.228880 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:51:38.228847    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:51:38.235823 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:51:38.235800    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:51:44.407395 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:51:44.407359    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:51:59.698323 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:51:59.698250    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:52:04.013272 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:04.013237    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:52:06.483210 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.477449    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:52:06.487606 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.487582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.490281 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.490258    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6ebb6a\"/\"rate-test-model-6ebb6a-kserve-self-signed-certs\""
Jul 03 14:52:06.491777 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.491758    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:52:06.543677 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543653    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.543794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.543794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543708    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.543794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543734    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr5qk\" (UniqueName: \"kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.543794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543774    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.543941 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.543812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645058 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645029    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645068    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr5qk\" (UniqueName: \"kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645088    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645251 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645522 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645505    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645591 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645536    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.645653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.645587    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.647542 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.647521    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.647807 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.647791    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.655163 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.655137    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr5qk\" (UniqueName: \"kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk\") pod \"rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") " pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.798176 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.798088    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:06.936450 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:06.936418    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:52:06.938736 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:52:06.938698    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9aba17a_9707_445b_beb6_831af2ed6883.slice/crio-4dc248d42cdaf41ae3dc2abfe494017a786ec25361e185ab82de7c08d54068c6 WatchSource:0}: Error finding container 4dc248d42cdaf41ae3dc2abfe494017a786ec25361e185ab82de7c08d54068c6: Status 404 returned error can't find the container with id 4dc248d42cdaf41ae3dc2abfe494017a786ec25361e185ab82de7c08d54068c6
Jul 03 14:52:07.362967 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:07.362923    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" event={"ID":"e9aba17a-9707-445b-beb6-831af2ed6883","Type":"ContainerStarted","Data":"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"}
Jul 03 14:52:07.362967 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:07.362970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" event={"ID":"e9aba17a-9707-445b-beb6-831af2ed6883","Type":"ContainerStarted","Data":"4dc248d42cdaf41ae3dc2abfe494017a786ec25361e185ab82de7c08d54068c6"}
Jul 03 14:52:07.385313 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:07.385263    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" podStartSLOduration=1.385247078 podStartE2EDuration="1.385247078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:52:07.382494722 +0000 UTC m=+2429.796391172" watchObservedRunningTime="2026-07-03 14:52:07.385247078 +0000 UTC m=+2429.799143528"
Jul 03 14:52:14.718237 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:14.718171    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" podUID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" containerName="main" containerID="cri-o://b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987" gracePeriod=45
Jul 03 14:52:14.977613 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:14.977559    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:52:15.018036 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018002    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018049    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018071    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018220 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018199    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ll7x\" (UniqueName: \"kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018225    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm\") pod \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\" (UID: \"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b\") "
Jul 03 14:52:15.018442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018417    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home" (OuterVolumeSpecName: "home") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:52:15.018625 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018596    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-home\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.018979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018934    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:52:15.018979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.018950    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:52:15.020757 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.020735    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm" (OuterVolumeSpecName: "dshm") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:52:15.020902 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.020886    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x" (OuterVolumeSpecName: "kube-api-access-8ll7x") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "kube-api-access-8ll7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:52:15.021157 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.021144    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" (UID: "e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:52:15.119979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.119937    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-model-cache\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.119979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.119974    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ll7x\" (UniqueName: \"kubernetes.io/projected/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-kube-api-access-8ll7x\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.119979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.119984    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-dshm\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.119979 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.119992    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tls-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.120259 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.120001    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b-tmp-dir\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:52:15.400997 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.400966    2561 generic.go:356] "Generic (PLEG): container finished" podID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" containerID="b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987" exitCode=0
Jul 03 14:52:15.401202 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.401013    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"
Jul 03 14:52:15.401202 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.401041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" event={"ID":"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b","Type":"ContainerDied","Data":"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"}
Jul 03 14:52:15.401202 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.401079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84" event={"ID":"e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b","Type":"ContainerDied","Data":"0429c373898c518cd88051dbf4ac86c05a90a2933a0397e18a87b6b1152fd01b"}
Jul 03 14:52:15.401202 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.401100    2561 scope.go:117] "RemoveContainer" containerID="b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"
Jul 03 14:52:15.410324 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.410305    2561 scope.go:117] "RemoveContainer" containerID="b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"
Jul 03 14:52:15.410569 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:52:15.410550    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987\": container with ID starting with b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987 not found: ID does not exist" containerID="b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"
Jul 03 14:52:15.410640 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.410578    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987"} err="failed to get container status \"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987\": rpc error: code = NotFound desc = could not find container \"b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987\": container with ID starting with b8a83dfc6ca482f65bf9adb5743bb2f538a321dbd77bf158cb3c529ff9724987 not found: ID does not exist"
Jul 03 14:52:15.425359 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.425333    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:52:15.429972 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:15.429951    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/sub-test-model-6ebb6a-kserve-774874cc4c-98k84"]
Jul 03 14:52:16.190976 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:16.190943    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" path="/var/lib/kubelet/pods/e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b/volumes"
Jul 03 14:52:16.798527 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:16.798494    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:16.798719 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:16.798564    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:16.810939 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:16.810917    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:17.429201 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:17.429152    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:52:18.899226 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:18.899189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:52:33.522756 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:33.522719    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:52:47.690852 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:52:47.690819    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:53:02.399761 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:02.399726    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:53:09.428851 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:09.428815    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:53:12.052172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.052137    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:53:12.052989 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.052951    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" podUID="36dfd345-d7a3-43fc-8be2-926634f9bc96" containerName="maas-api" containerID="cri-o://418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99" gracePeriod=30
Jul 03 14:53:12.308701 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.308639    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:53:12.447786 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.447754    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") pod \"36dfd345-d7a3-43fc-8be2-926634f9bc96\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") "
Jul 03 14:53:12.447947 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.447810    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh74x\" (UniqueName: \"kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x\") pod \"36dfd345-d7a3-43fc-8be2-926634f9bc96\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") "
Jul 03 14:53:12.447947 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.447934    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca\") pod \"36dfd345-d7a3-43fc-8be2-926634f9bc96\" (UID: \"36dfd345-d7a3-43fc-8be2-926634f9bc96\") "
Jul 03 14:53:12.448361 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.448337    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36dfd345-d7a3-43fc-8be2-926634f9bc96" (UID: "36dfd345-d7a3-43fc-8be2-926634f9bc96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:53:12.450023 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.450003    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36dfd345-d7a3-43fc-8be2-926634f9bc96" (UID: "36dfd345-d7a3-43fc-8be2-926634f9bc96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:53:12.450071 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.450055    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x" (OuterVolumeSpecName: "kube-api-access-wh74x") pod "36dfd345-d7a3-43fc-8be2-926634f9bc96" (UID: "36dfd345-d7a3-43fc-8be2-926634f9bc96"). InnerVolumeSpecName "kube-api-access-wh74x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:53:12.549250 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.549220    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36dfd345-d7a3-43fc-8be2-926634f9bc96-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:12.549250 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.549247    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh74x\" (UniqueName: \"kubernetes.io/projected/36dfd345-d7a3-43fc-8be2-926634f9bc96-kube-api-access-wh74x\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:12.549250 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.549258    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36dfd345-d7a3-43fc-8be2-926634f9bc96-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:12.697955 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.697918    2561 generic.go:356] "Generic (PLEG): container finished" podID="36dfd345-d7a3-43fc-8be2-926634f9bc96" containerID="418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99" exitCode=0
Jul 03 14:53:12.698117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.697963    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"
Jul 03 14:53:12.698117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.697999    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" event={"ID":"36dfd345-d7a3-43fc-8be2-926634f9bc96","Type":"ContainerDied","Data":"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"}
Jul 03 14:53:12.698117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.698054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl" event={"ID":"36dfd345-d7a3-43fc-8be2-926634f9bc96","Type":"ContainerDied","Data":"3f3c698999f6455800e1894560ae46753d5778860a020e21defb3d5f07ca2f45"}
Jul 03 14:53:12.698117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.698083    2561 scope.go:117] "RemoveContainer" containerID="418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"
Jul 03 14:53:12.707784 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.707766    2561 scope.go:117] "RemoveContainer" containerID="418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"
Jul 03 14:53:12.708064 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:53:12.708039    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99\": container with ID starting with 418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99 not found: ID does not exist" containerID="418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"
Jul 03 14:53:12.708115 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.708078    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99"} err="failed to get container status \"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99\": rpc error: code = NotFound desc = could not find container \"418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99\": container with ID starting with 418a43541e9ca425cf8788bc5c50fb81030bd996abc45bbeb1f0f43dfd1a0c99 not found: ID does not exist"
Jul 03 14:53:12.720858 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.720833    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:53:12.725283 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:12.725264    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6ebb6a-866cc984d9-tjbbl"]
Jul 03 14:53:14.192371 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:14.192341    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36dfd345-d7a3-43fc-8be2-926634f9bc96" path="/var/lib/kubelet/pods/36dfd345-d7a3-43fc-8be2-926634f9bc96/volumes"
Jul 03 14:53:15.498504 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:15.498468    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:53:24.448387 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.448339    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" podUID="e9aba17a-9707-445b-beb6-831af2ed6883" containerName="main" containerID="cri-o://cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0" gracePeriod=45
Jul 03 14:53:24.702692 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.702638    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:53:24.755457 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.755423    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9aba17a-9707-445b-beb6-831af2ed6883" containerID="cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0" exitCode=0
Jul 03 14:53:24.755587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.755466    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"
Jul 03 14:53:24.755587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.755513    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" event={"ID":"e9aba17a-9707-445b-beb6-831af2ed6883","Type":"ContainerDied","Data":"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"}
Jul 03 14:53:24.755587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.755557    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv" event={"ID":"e9aba17a-9707-445b-beb6-831af2ed6883","Type":"ContainerDied","Data":"4dc248d42cdaf41ae3dc2abfe494017a786ec25361e185ab82de7c08d54068c6"}
Jul 03 14:53:24.755587 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.755574    2561 scope.go:117] "RemoveContainer" containerID="cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"
Jul 03 14:53:24.759991 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.759964    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760028    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr5qk\" (UniqueName: \"kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760253    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760279 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760266    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:24.760390 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760323    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs\") pod \"e9aba17a-9707-445b-beb6-831af2ed6883\" (UID: \"e9aba17a-9707-445b-beb6-831af2ed6883\") "
Jul 03 14:53:24.760390 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760329    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home" (OuterVolumeSpecName: "home") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:24.760528 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760501    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:24.760698 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760666    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-model-cache\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:24.760698 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760692    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-tmp-dir\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:24.760809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.760709    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-home\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:24.762520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.762497    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:53:24.762679 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.762656    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm" (OuterVolumeSpecName: "dshm") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:24.762763 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.762741    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk" (OuterVolumeSpecName: "kube-api-access-fr5qk") pod "e9aba17a-9707-445b-beb6-831af2ed6883" (UID: "e9aba17a-9707-445b-beb6-831af2ed6883"). InnerVolumeSpecName "kube-api-access-fr5qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:53:24.764844 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.764825    2561 scope.go:117] "RemoveContainer" containerID="cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"
Jul 03 14:53:24.765108 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:53:24.765090    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0\": container with ID starting with cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0 not found: ID does not exist" containerID="cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"
Jul 03 14:53:24.765172 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.765120    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0"} err="failed to get container status \"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0\": rpc error: code = NotFound desc = could not find container \"cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0\": container with ID starting with cc5dcd7208027d111d24065bd3f419c4fc6774418efad82b35fe17307ebd81d0 not found: ID does not exist"
Jul 03 14:53:24.861330 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.861297    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9aba17a-9707-445b-beb6-831af2ed6883-dshm\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:24.861330 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.861326    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9aba17a-9707-445b-beb6-831af2ed6883-tls-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:24.861497 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:24.861339    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr5qk\" (UniqueName: \"kubernetes.io/projected/e9aba17a-9707-445b-beb6-831af2ed6883-kube-api-access-fr5qk\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:25.077518 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:25.077490    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:53:25.081601 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:25.081577    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6ebb6a/rate-test-model-6ebb6a-kserve-6c984c8fdc-br7pv"]
Jul 03 14:53:26.191071 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:26.191036    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9aba17a-9707-445b-beb6-831af2ed6883" path="/var/lib/kubelet/pods/e9aba17a-9707-445b-beb6-831af2ed6883/volumes"
Jul 03 14:53:54.487653 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:54.487621    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:53:54.488092 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:54.487839    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" containerID="cri-o://74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703" gracePeriod=30
Jul 03 14:53:55.631200 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:55.631145    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:53:55.631670 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:55.631423    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerName="maas-api" containerID="cri-o://a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb" gracePeriod=30
Jul 03 14:53:56.078534 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.078509    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:53:56.244307 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.244237    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls\") pod \"b90be159-8f67-4197-81ee-dc41a5f8e941\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") "
Jul 03 14:53:56.244307 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.244278    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjjtx\" (UniqueName: \"kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx\") pod \"b90be159-8f67-4197-81ee-dc41a5f8e941\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") "
Jul 03 14:53:56.244497 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.244325    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca\") pod \"b90be159-8f67-4197-81ee-dc41a5f8e941\" (UID: \"b90be159-8f67-4197-81ee-dc41a5f8e941\") "
Jul 03 14:53:56.244755 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.244724    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b90be159-8f67-4197-81ee-dc41a5f8e941" (UID: "b90be159-8f67-4197-81ee-dc41a5f8e941"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:53:56.246261 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.246234    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b90be159-8f67-4197-81ee-dc41a5f8e941" (UID: "b90be159-8f67-4197-81ee-dc41a5f8e941"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:53:56.246409 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.246387    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx" (OuterVolumeSpecName: "kube-api-access-wjjtx") pod "b90be159-8f67-4197-81ee-dc41a5f8e941" (UID: "b90be159-8f67-4197-81ee-dc41a5f8e941"). InnerVolumeSpecName "kube-api-access-wjjtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:53:56.345958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.345936    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b90be159-8f67-4197-81ee-dc41a5f8e941-maas-api-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:56.345958 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.345959    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjjtx\" (UniqueName: \"kubernetes.io/projected/b90be159-8f67-4197-81ee-dc41a5f8e941-kube-api-access-wjjtx\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:56.346112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.345970    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90be159-8f67-4197-81ee-dc41a5f8e941-openshift-service-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:56.915078 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.915047    2561 generic.go:356] "Generic (PLEG): container finished" podID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerID="a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb" exitCode=0
Jul 03 14:53:56.915520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.915107    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"
Jul 03 14:53:56.915520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.915132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" event={"ID":"b90be159-8f67-4197-81ee-dc41a5f8e941","Type":"ContainerDied","Data":"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"}
Jul 03 14:53:56.915520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.915222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" event={"ID":"b90be159-8f67-4197-81ee-dc41a5f8e941","Type":"ContainerDied","Data":"04caa7ca74a7deb66ac8de1f3cb2d19d0b15fcf8343d32d2d75e1e67b5e8f6cd"}
Jul 03 14:53:56.915520 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.915246    2561 scope.go:117] "RemoveContainer" containerID="a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"
Jul 03 14:53:56.924673 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.924658    2561 scope.go:117] "RemoveContainer" containerID="a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"
Jul 03 14:53:56.924936 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:53:56.924912    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb\": container with ID starting with a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb not found: ID does not exist" containerID="a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"
Jul 03 14:53:56.925034 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.924941    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb"} err="failed to get container status \"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb\": rpc error: code = NotFound desc = could not find container \"a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb\": container with ID starting with a3fb267fcf162f5d4ef56244ce5dd28d85985525bc24c55ddafc20d1161fc0cb not found: ID does not exist"
Jul 03 14:53:56.942900 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.942873    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:53:56.948014 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:56.947994    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d"]
Jul 03 14:53:58.190855 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:58.190825    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" path="/var/lib/kubelet/pods/b90be159-8f67-4197-81ee-dc41a5f8e941/volumes"
Jul 03 14:53:58.312940 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:58.312903    2561 patch_prober.go:28] interesting pod/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 03 14:53:58.313089 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:58.312953    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 03 14:53:58.700268 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:58.700172    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pjbw7"]
Jul 03 14:53:59.046239 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.046118    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-b-1dd877-6d7dfc9f45-h524d" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": context deadline exceeded"
Jul 03 14:53:59.741234 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.741205    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:53:59.880632 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880605    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dxwv\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880632 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880637    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880656    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880733    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880756    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880772    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880805    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880850    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.880899 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880884    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket\") pod \"277e3aef-4df5-49b5-a469-1e8912a8f443\" (UID: \"277e3aef-4df5-49b5-a469-1e8912a8f443\") "
Jul 03 14:53:59.881300 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.880979    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:59.881300 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.881142    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:59.881300 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.881228    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:59.881300 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.881299    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.881671 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.881316    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.881671 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.881627    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:53:59.882066 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.882032    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data" (OuterVolumeSpecName: "istio-data") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:59.882975 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.882946    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv" (OuterVolumeSpecName: "kube-api-access-2dxwv") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "kube-api-access-2dxwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:53:59.883099 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.883005    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token" (OuterVolumeSpecName: "istio-token") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:53:59.883164 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.883112    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:53:59.883262 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.883240    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "277e3aef-4df5-49b5-a469-1e8912a8f443" (UID: "277e3aef-4df5-49b5-a469-1e8912a8f443"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:53:59.932273 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.932249    2561 generic.go:356] "Generic (PLEG): container finished" podID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerID="74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703" exitCode=0
Jul 03 14:53:59.932391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.932296    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"
Jul 03 14:53:59.932391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.932311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerDied","Data":"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"}
Jul 03 14:53:59.932391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.932360    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd" event={"ID":"277e3aef-4df5-49b5-a469-1e8912a8f443","Type":"ContainerDied","Data":"957442d2369a992ece70512afe2f846c26b8bb586997089a55ad30e935d328df"}
Jul 03 14:53:59.932391 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.932386    2561 scope.go:117] "RemoveContainer" containerID="74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"
Jul 03 14:53:59.941296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.941277    2561 scope.go:117] "RemoveContainer" containerID="97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"
Jul 03 14:53:59.948607 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.948592    2561 scope.go:117] "RemoveContainer" containerID="74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"
Jul 03 14:53:59.948840 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:53:59.948810    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703\": container with ID starting with 74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703 not found: ID does not exist" containerID="74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"
Jul 03 14:53:59.948921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.948838    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703"} err="failed to get container status \"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703\": rpc error: code = NotFound desc = could not find container \"74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703\": container with ID starting with 74aa190732fdada985734246405a6e6a6246390d32b1795746cbba29ccc6a703 not found: ID does not exist"
Jul 03 14:53:59.948921 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.948855    2561 scope.go:117] "RemoveContainer" containerID="97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"
Jul 03 14:53:59.949094 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:53:59.949077    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55\": container with ID starting with 97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55 not found: ID does not exist" containerID="97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"
Jul 03 14:53:59.949134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.949100    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55"} err="failed to get container status \"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55\": rpc error: code = NotFound desc = could not find container \"97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55\": container with ID starting with 97802a482774b748938dab6745eb3d3d1e0c3fe0d2630cdf10416fbf7e66bc55 not found: ID does not exist"
Jul 03 14:53:59.958269 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.958248    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:53:59.972016 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.971997    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6ebb6a-openshift-default-675b6d7c8c-b4sqd"]
Jul 03 14:53:59.982442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982422    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982442 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982440    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982450    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982460    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/277e3aef-4df5-49b5-a469-1e8912a8f443-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982468    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/277e3aef-4df5-49b5-a469-1e8912a8f443-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982475    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dxwv\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-kube-api-access-2dxwv\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:53:59.982549 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:53:59.982483    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/277e3aef-4df5-49b5-a469-1e8912a8f443-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:00.190916 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:00.190837    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" path="/var/lib/kubelet/pods/277e3aef-4df5-49b5-a469-1e8912a8f443/volumes"
Jul 03 14:54:02.422516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:02.422478    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:54:02.422997 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:02.422691    2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy" containerID="cri-o://c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13" gracePeriod=30
Jul 03 14:54:07.642637 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.642610    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lgl58_b1a21974-4c30-4393-8b1f-cbced2867257/manager/0.log"
Jul 03 14:54:07.676841 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.676819    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:54:07.755836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755751    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.755836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755797    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.755836 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755832    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755945    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755963    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmnln\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.755981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket\") pod \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\" (UID: \"dfabebd2-27e2-41b3-991f-5a5a6a3253f1\") "
Jul 03 14:54:07.756435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756124    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:54:07.756435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756175    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:54:07.756435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756378    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-certs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.756435 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756396    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-workload-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.756651 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756439    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:54:07.756651 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.756460    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:54:07.757235 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.757206    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:54:07.758199 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.758163    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:54:07.758296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.758279    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 03 14:54:07.758340 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.758322    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln" (OuterVolumeSpecName: "kube-api-access-fmnln") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "kube-api-access-fmnln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:54:07.758426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.758411    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "dfabebd2-27e2-41b3-991f-5a5a6a3253f1" (UID: "dfabebd2-27e2-41b3-991f-5a5a6a3253f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:54:07.857136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857108    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istiod-ca-cert\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857132    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857143    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-podinfo\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857151    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmnln\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-kube-api-access-fmnln\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857160    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857168    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-credential-socket\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.857370 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.857176    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfabebd2-27e2-41b3-991f-5a5a6a3253f1-istio-envoy\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\""
Jul 03 14:54:07.974001 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.973971    2561 generic.go:356] "Generic (PLEG): container finished" podID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerID="c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13" exitCode=0
Jul 03 14:54:07.974230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.974017    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"
Jul 03 14:54:07.974230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.974064    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerDied","Data":"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"}
Jul 03 14:54:07.974230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.974123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8" event={"ID":"dfabebd2-27e2-41b3-991f-5a5a6a3253f1","Type":"ContainerDied","Data":"56417980be8350714b822f90497f38eac3115998b91cc511846570d25b28a49c"}
Jul 03 14:54:07.974230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.974152    2561 scope.go:117] "RemoveContainer" containerID="c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"
Jul 03 14:54:07.984095 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.984079    2561 scope.go:117] "RemoveContainer" containerID="176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"
Jul 03 14:54:07.992009 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.991993    2561 scope.go:117] "RemoveContainer" containerID="c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"
Jul 03 14:54:07.992260 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:54:07.992244    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13\": container with ID starting with c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13 not found: ID does not exist" containerID="c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"
Jul 03 14:54:07.992304 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.992268    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13"} err="failed to get container status \"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13\": rpc error: code = NotFound desc = could not find container \"c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13\": container with ID starting with c45cb4ae34bd986057d633ce6900bda7ade15517515b6daad247e1583edb9a13 not found: ID does not exist"
Jul 03 14:54:07.992304 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.992284    2561 scope.go:117] "RemoveContainer" containerID="176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"
Jul 03 14:54:07.992529 ip-10-0-133-223 kubenswrapper[2561]: E0703 14:54:07.992511    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a\": container with ID starting with 176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a not found: ID does not exist" containerID="176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"
Jul 03 14:54:07.992562 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.992536    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a"} err="failed to get container status \"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a\": rpc error: code = NotFound desc = could not find container \"176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a\": container with ID starting with 176113c93a80d8d9ab16cfd572866f28ee6f22de84d6ffdc04c002a6f5c19d3a not found: ID does not exist"
Jul 03 14:54:07.999022 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:07.999000    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:54:08.003429 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:08.003409    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718150-7rwdn_15731c11-6060-49a8-8b6f-e89c5f0a3b47/cleanup/0.log"
Jul 03 14:54:08.004789 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:08.004772    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1dd877-openshift-default-7fb98cd9b7-cbxb8"]
Jul 03 14:54:08.191371 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:08.191341    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" path="/var/lib/kubelet/pods/dfabebd2-27e2-41b3-991f-5a5a6a3253f1/volumes"
Jul 03 14:54:08.743490 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:08.743461    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6554f4cb-7tp42_077be1f8-be6e-4f15-83a8-4ffb40d2e209/copy-manifests/0.log"
Jul 03 14:54:08.755785 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:08.755757    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6554f4cb-7tp42_077be1f8-be6e-4f15-83a8-4ffb40d2e209/manager/0.log"
Jul 03 14:54:11.089437 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:11.089412    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pjbw7_71412083-3365-4fce-9da6-599c6c11879f/limitador/0.log"
Jul 03 14:54:11.236641 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:11.236604    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k9csf_c4e4789a-78e2-48d0-b81d-106cd3c7e116/manager/0.log"
Jul 03 14:54:11.593873 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:11.593841    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tv4zj_1471b33a-a9f4-4431-af37-fd5ca96ab1df/istio-proxy/0.log"
Jul 03 14:54:13.148525 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:13.148484    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzqh2d_324d87c6-a20e-428a-a590-4ff014cd18ac/main/0.log"
Jul 03 14:54:13.370299 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:13.370263    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-n687h_a3e0df26-3f94-48b9-a5b4-38195e276ce4/main/0.log"
Jul 03 14:54:17.616833 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.616798    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-29ndn/must-gather-sdgdx"]
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617154    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9aba17a-9707-445b-beb6-831af2ed6883" containerName="main"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617166    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9aba17a-9707-445b-beb6-831af2ed6883" containerName="main"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617196    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617204    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617215    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36dfd345-d7a3-43fc-8be2-926634f9bc96" containerName="maas-api"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617222    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36dfd345-d7a3-43fc-8be2-926634f9bc96" containerName="maas-api"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617228    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617234    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617236 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617241    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerName="maas-api"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617246    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerName="maas-api"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617259    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617264    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617272    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617277    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617282    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" containerName="main"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617287    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" containerName="main"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617343    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617351    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="277e3aef-4df5-49b5-a469-1e8912a8f443" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617356    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5affb4a-04e8-42a7-a2c9-fb9f52e86d8b" containerName="main"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617362    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36dfd345-d7a3-43fc-8be2-926634f9bc96" containerName="maas-api"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617367    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617372    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b90be159-8f67-4197-81ee-dc41a5f8e941" containerName="maas-api"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617376    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfabebd2-27e2-41b3-991f-5a5a6a3253f1" containerName="istio-proxy"
Jul 03 14:54:17.617516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.617381    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9aba17a-9707-445b-beb6-831af2ed6883" containerName="main"
Jul 03 14:54:17.620443 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.620425    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.622831 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.622814    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-29ndn\"/\"openshift-service-ca.crt\""
Jul 03 14:54:17.622941 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.622919    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-29ndn\"/\"kube-root-ca.crt\""
Jul 03 14:54:17.623670 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.623650    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-29ndn\"/\"default-dockercfg-cz7zd\""
Jul 03 14:54:17.631589 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.631569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-29ndn/must-gather-sdgdx"]
Jul 03 14:54:17.744639 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.744606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1779394e-735b-4360-a2ef-f63938e55e1b-must-gather-output\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.744794 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.744671    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s985h\" (UniqueName: \"kubernetes.io/projected/1779394e-735b-4360-a2ef-f63938e55e1b-kube-api-access-s985h\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.845425 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.845387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s985h\" (UniqueName: \"kubernetes.io/projected/1779394e-735b-4360-a2ef-f63938e55e1b-kube-api-access-s985h\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.845667 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.845433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1779394e-735b-4360-a2ef-f63938e55e1b-must-gather-output\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.845750 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.845720    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1779394e-735b-4360-a2ef-f63938e55e1b-must-gather-output\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.855123 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.855104    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s985h\" (UniqueName: \"kubernetes.io/projected/1779394e-735b-4360-a2ef-f63938e55e1b-kube-api-access-s985h\") pod \"must-gather-sdgdx\" (UID: \"1779394e-735b-4360-a2ef-f63938e55e1b\") " pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:17.929330 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:17.929274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-29ndn/must-gather-sdgdx"
Jul 03 14:54:18.064053 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:18.064026    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-29ndn/must-gather-sdgdx"]
Jul 03 14:54:18.066860 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:54:18.066834    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1779394e_735b_4360_a2ef_f63938e55e1b.slice/crio-a58e95b27e88197188e846ff1822db44c551e16d678205748985a14aaf717f91 WatchSource:0}: Error finding container a58e95b27e88197188e846ff1822db44c551e16d678205748985a14aaf717f91: Status 404 returned error can't find the container with id a58e95b27e88197188e846ff1822db44c551e16d678205748985a14aaf717f91
Jul 03 14:54:18.068809 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:18.068788    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:54:19.037136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:19.037040    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-29ndn/must-gather-sdgdx" event={"ID":"1779394e-735b-4360-a2ef-f63938e55e1b","Type":"ContainerStarted","Data":"4a60eba004dc1b8ab157338859ac29bd3610396d08be2cf2c29cfa7c2ab8fd0f"}
Jul 03 14:54:19.037136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:19.037102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-29ndn/must-gather-sdgdx" event={"ID":"1779394e-735b-4360-a2ef-f63938e55e1b","Type":"ContainerStarted","Data":"9e716b514969c9e6720b74a96fa1acb1a62ab9b3817497738e150f73a6e02370"}
Jul 03 14:54:19.037136 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:19.037121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-29ndn/must-gather-sdgdx" event={"ID":"1779394e-735b-4360-a2ef-f63938e55e1b","Type":"ContainerStarted","Data":"a58e95b27e88197188e846ff1822db44c551e16d678205748985a14aaf717f91"}
Jul 03 14:54:20.527156 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:20.527122    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dmdfr_07fdca58-6749-4863-8fcd-e302905155b2/global-pull-secret-syncer/0.log"
Jul 03 14:54:20.681119 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:20.681091    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-clz2s_1078bd6c-b3c3-46e9-95e6-8e44a0561a95/konnectivity-agent/0.log"
Jul 03 14:54:20.750834 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:20.750776    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-223.ec2.internal_0e1adb89e96ffa1aea4eb2057cc3c519/haproxy/0.log"
Jul 03 14:54:26.070760 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:26.070732    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pjbw7_71412083-3365-4fce-9da6-599c6c11879f/limitador/0.log"
Jul 03 14:54:26.158357 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:26.158325    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k9csf_c4e4789a-78e2-48d0-b81d-106cd3c7e116/manager/0.log"
Jul 03 14:54:28.175131 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.175090    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d9546c774-4rpkx_82f7d731-45fd-44f8-8eb2-e638d58bcab5/metrics-server/0.log"
Jul 03 14:54:28.213985 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.213949    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-tzqrn_f5da2aa3-df10-43c5-8c4f-0d945a8dffc3/monitoring-plugin/0.log"
Jul 03 14:54:28.339385 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.339353    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sbkm_00137202-1f2e-4cea-bebf-f5c00e0ac8ea/node-exporter/0.log"
Jul 03 14:54:28.366516 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.366479    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sbkm_00137202-1f2e-4cea-bebf-f5c00e0ac8ea/kube-rbac-proxy/0.log"
Jul 03 14:54:28.393610 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.393584    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sbkm_00137202-1f2e-4cea-bebf-f5c00e0ac8ea/init-textfile/0.log"
Jul 03 14:54:28.650856 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.650780    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/prometheus/0.log"
Jul 03 14:54:28.680847 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.680815    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/config-reloader/0.log"
Jul 03 14:54:28.712112 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.712056    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/thanos-sidecar/0.log"
Jul 03 14:54:28.743117 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.743088    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/kube-rbac-proxy-web/0.log"
Jul 03 14:54:28.771269 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.771241    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/kube-rbac-proxy/0.log"
Jul 03 14:54:28.797828 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.797786    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/kube-rbac-proxy-thanos/0.log"
Jul 03 14:54:28.824065 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:28.824029    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_86435dba-2559-4b14-963d-df5147012437/init-config-reloader/0.log"
Jul 03 14:54:29.033978 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.033895    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/thanos-query/0.log"
Jul 03 14:54:29.066303 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.066274    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/kube-rbac-proxy-web/0.log"
Jul 03 14:54:29.092853 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.092826    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/kube-rbac-proxy/0.log"
Jul 03 14:54:29.123906 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.123875    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/prom-label-proxy/0.log"
Jul 03 14:54:29.153397 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.153368    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/kube-rbac-proxy-rules/0.log"
Jul 03 14:54:29.181865 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:29.181839    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-657b7cbf4b-hk7fz_734808b7-e7c0-438b-a60b-b7568f784d9b/kube-rbac-proxy-metrics/0.log"
Jul 03 14:54:30.103995 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.103936    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-29ndn/must-gather-sdgdx" podStartSLOduration=12.462914722 podStartE2EDuration="13.103888189s" totalImagesPullingTime="640.973467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:54:19.07170549 +0000 UTC m=+2561.485601941" watchObservedRunningTime="2026-07-03 14:54:30.103888189 +0000 UTC m=+2572.517784634"
Jul 03 14:54:30.107103 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.107066    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"]
Jul 03 14:54:30.115842 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.115820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.122130 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.122104    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"]
Jul 03 14:54:30.180658 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.180627    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-proc\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.180857 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.180750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-sys\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.180934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.180880    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-lib-modules\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.180934 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.180907    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-podres\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.181041 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.180975    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv9rl\" (UniqueName: \"kubernetes.io/projected/ac24d9d2-161a-41fb-9962-35b27f340730-kube-api-access-dv9rl\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.282627 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.282590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv9rl\" (UniqueName: \"kubernetes.io/projected/ac24d9d2-161a-41fb-9962-35b27f340730-kube-api-access-dv9rl\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.283286 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.283246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-proc\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.283488 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.283370    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-proc\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.283617 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.283459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-sys\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.283817 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.283802    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-lib-modules\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.284072 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.284056    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-podres\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.284351 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.284016    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-lib-modules\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.284469 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.283525    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-sys\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.284555 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.284268    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac24d9d2-161a-41fb-9962-35b27f340730-podres\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.292713 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.292666    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv9rl\" (UniqueName: \"kubernetes.io/projected/ac24d9d2-161a-41fb-9962-35b27f340730-kube-api-access-dv9rl\") pod \"perf-node-gather-daemonset-n88pb\" (UID: \"ac24d9d2-161a-41fb-9962-35b27f340730\") " pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.434468 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.434348    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:30.628747 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:30.628721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"]
Jul 03 14:54:30.631265 ip-10-0-133-223 kubenswrapper[2561]: W0703 14:54:30.631236    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podac24d9d2_161a_41fb_9962_35b27f340730.slice/crio-f194c19ed88bb9063cb76078575e42ea17290ff8faabf3d03ad1e34e292d3897 WatchSource:0}: Error finding container f194c19ed88bb9063cb76078575e42ea17290ff8faabf3d03ad1e34e292d3897: Status 404 returned error can't find the container with id f194c19ed88bb9063cb76078575e42ea17290ff8faabf3d03ad1e34e292d3897
Jul 03 14:54:31.093280 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.093254    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/1.log"
Jul 03 14:54:31.100888 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.100856    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cj294_58d23fb7-3fe9-4b16-8719-a155bf7fa66c/console-operator/2.log"
Jul 03 14:54:31.140930 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.140888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb" event={"ID":"ac24d9d2-161a-41fb-9962-35b27f340730","Type":"ContainerStarted","Data":"791ed7fe717e1ceda0c9e2bd9470c1f4656740e7600b7a5cc885304165eed3b3"}
Jul 03 14:54:31.141122 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.140960    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb" event={"ID":"ac24d9d2-161a-41fb-9962-35b27f340730","Type":"ContainerStarted","Data":"f194c19ed88bb9063cb76078575e42ea17290ff8faabf3d03ad1e34e292d3897"}
Jul 03 14:54:31.141426 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.141273    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:31.161566 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:31.161501    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb" podStartSLOduration=1.161479113 podStartE2EDuration="1.161479113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:54:31.159593054 +0000 UTC m=+2573.573489508" watchObservedRunningTime="2026-07-03 14:54:31.161479113 +0000 UTC m=+2573.575375563"
Jul 03 14:54:32.304704 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:32.304668    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-mvg7r_c57f3681-87b3-4c03-ac6b-689fc2bf174b/volume-data-source-validator/0.log"
Jul 03 14:54:33.492631 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:33.492596    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hlpdf_406123bb-f445-4919-b537-dd4c8049c717/dns/0.log"
Jul 03 14:54:33.524744 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:33.524721    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hlpdf_406123bb-f445-4919-b537-dd4c8049c717/kube-rbac-proxy/0.log"
Jul 03 14:54:33.657109 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:33.657084    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-778zb_8e595515-da23-4809-b415-5f5ba0ee0014/dns-node-resolver/0.log"
Jul 03 14:54:34.283294 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:34.283262    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54fcd86df9-nt7sz_9eb6bb63-4bd8-4098-98d3-47fcb1faf5b2/registry/0.log"
Jul 03 14:54:34.363256 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:34.363229    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4c2sk_80a77bb5-d85b-4df5-b4ca-9f294ead9954/node-ca/0.log"
Jul 03 14:54:35.672369 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:35.672343    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tv4zj_1471b33a-a9f4-4431-af37-fd5ca96ab1df/istio-proxy/0.log"
Jul 03 14:54:37.103230 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:37.103199    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rmgm9_494eba7d-34c7-493e-8829-84bfaefb21d5/serve-healthcheck-canary/0.log"
Jul 03 14:54:37.163444 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:37.163405    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-29ndn/perf-node-gather-daemonset-n88pb"
Jul 03 14:54:37.733134 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:37.733077    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxv2_af900b5e-358a-481c-a359-9844bccc514c/kube-rbac-proxy/0.log"
Jul 03 14:54:37.762875 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:37.762851    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxv2_af900b5e-358a-481c-a359-9844bccc514c/exporter/0.log"
Jul 03 14:54:37.810207 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:37.810159    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxv2_af900b5e-358a-481c-a359-9844bccc514c/extractor/0.log"
Jul 03 14:54:40.598417 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:40.598382    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lgl58_b1a21974-4c30-4393-8b1f-cbced2867257/manager/0.log"
Jul 03 14:54:40.783337 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:40.783309    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tv4zj_1471b33a-a9f4-4431-af37-fd5ca96ab1df/istio-proxy/0.log"
Jul 03 14:54:40.917825 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:40.917751    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718150-7rwdn_15731c11-6060-49a8-8b6f-e89c5f0a3b47/cleanup/0.log"
Jul 03 14:54:41.448326 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:41.448220    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6554f4cb-7tp42_077be1f8-be6e-4f15-83a8-4ffb40d2e209/manager/0.log"
Jul 03 14:54:41.470952 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:41.470928    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6554f4cb-7tp42_077be1f8-be6e-4f15-83a8-4ffb40d2e209/copy-manifests/0.log"
Jul 03 14:54:42.330506 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:42.330442    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tv4zj_1471b33a-a9f4-4431-af37-fd5ca96ab1df/istio-proxy/0.log"
Jul 03 14:54:42.967296 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:42.967172    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77cf67f8d7-ggdxv_564f2955-28cb-4b69-a4ef-0e6446705fb8/manager/0.log"
Jul 03 14:54:43.000336 ip-10-0-133-223 kubenswrapper[2561]: I0703 14:54:43.000309    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lfwf8_61dcfe99-6bbb-442c-a77a-7c2674fb789d/openshift-lws-operator/0.log"