Jul 07 19:09:06.682096 ip-10-0-136-217 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 19:09:07.006862 ip-10-0-136-217 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 19:09:07.006862 ip-10-0-136-217 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 19:09:07.006862 ip-10-0-136-217 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 19:09:07.006862 ip-10-0-136-217 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 19:09:07.006862 ip-10-0-136-217 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 19:09:07.008397 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.008315    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 19:09:07.013264 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013233    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.013264 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013262    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.013264 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013267    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.013264 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013272    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013278    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013282    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013287    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013291    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013295    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013306    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013311    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013315    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013319    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.013489 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013409    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013593    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013602    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013607    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013610    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013612    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013615    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013618    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013620    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013623    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013626    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013629    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013631    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013634    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013637    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013639    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013642    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013644    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013647    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013650    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.013862 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013653    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013655    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013660    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013663    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013666    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013668    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013671    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013673    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013676    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013683    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013686    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013689    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013692    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013695    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013698    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013700    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013703    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013705    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013708    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013711    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.014509 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013714    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013718    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013722    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013725    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013728    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013731    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013734    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013736    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013740    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013744    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013747    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013750    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013754    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013757    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013759    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013762    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013764    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013767    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013769    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.015038 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013772    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013775    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013777    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013780    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013782    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013789    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013793    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013796    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013799    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013802    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013805    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013807    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013810    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013813    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013815    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013818    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013820    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013823    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013825    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.015492 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013828    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013848    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013852    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013854    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013857    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013859    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013863    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.013865    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014592    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014599    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014602    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014605    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014607    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014610    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014612    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014615    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014619    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014621    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014624    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014627    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.015997 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014629    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014632    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014634    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014637    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014641    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014651    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014654    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014657    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014659    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014662    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014664    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014667    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014669    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014672    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014674    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014677    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014680    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014682    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014684    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.016501 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014687    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014689    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014692    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014694    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014696    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014699    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014702    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014704    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014707    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014710    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014713    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014715    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014718    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014722    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014725    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014728    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014731    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014733    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014736    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.016995 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014745    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014747    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014750    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014752    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014755    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014757    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014759    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014762    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014764    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014767    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014769    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014772    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014774    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014776    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014779    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014781    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014784    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014786    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014789    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014792    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.017472 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014795    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014797    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014800    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014802    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014805    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014807    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014809    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014812    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014814    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014817    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014819    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014821    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014826    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014855    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014859    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014862    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014865    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014867    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014870    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.017981 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014872    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014875    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014878    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014880    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014882    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014885    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014887    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014890    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014892    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.014895    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015290    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015295    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015298    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015300    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015303    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015306    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015309    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015311    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015314    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015316    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.018468 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015318    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015321    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015325    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015329    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015332    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015335    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015344    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015347    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015350    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015352    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015355    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015357    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015360    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015362    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015365    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015368    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015370    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015373    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015375    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.018967 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015378    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015380    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015382    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015385    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015388    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015391    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015393    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015396    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015398    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015401    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015403    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015405    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015408    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015410    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015413    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015415    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015418    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015420    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015423    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015425    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.019419 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015434    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015437    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015439    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015442    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015444    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015447    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015451    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015454    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015460    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015462    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015465    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015467    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015470    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015473    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015475    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015478    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015480    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015482    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015486    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.019994 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015490    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015492    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015496    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015499    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015502    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015505    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015508    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015511    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015513    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015516    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015518    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015521    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015523    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015525    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015534    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015537    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015539    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015542    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015544    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015547    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.020465 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015549    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015555    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015557    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015560    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015562    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015564    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015567    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015570    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015572    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015575    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.015577    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015650    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015657    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015666    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015670    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015673    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015676    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015680    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015684    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015687    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015690    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 19:09:07.020959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015693    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015696    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015699    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015702    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015705    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015708    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015717    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015720    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015727    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015730    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015733    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015736    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015742    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015746    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015749    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015751    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015754    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015757    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015760    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015763    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015766    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015769    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015773    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015776    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015778    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 19:09:07.021498 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015781    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015784    2579 flags.go:64] FLAG: --enable-server="true"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015787    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015794    2579 flags.go:64] FLAG: --event-burst="100"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015796    2579 flags.go:64] FLAG: --event-qps="50"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015799    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015802    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015805    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015808    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015811    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015813    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015816    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015819    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015822    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015846    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015851    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015853    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015870    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015874    2579 flags.go:64] FLAG: --feature-gates=""
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015880    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015884    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015887    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015890    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015893    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015896    2579 flags.go:64] FLAG: --help="false"
Jul 07 19:09:07.022113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015898    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015901    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015904    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015907    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015910    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015913    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015916    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015919    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015921    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015924    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015927    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015930    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015932    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015935    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015938    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015940    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015943    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015946    2579 flags.go:64] FLAG: --lock-file=""
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015948    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015951    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015954    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015959    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015967    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015971    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 19:09:07.022747 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015973    2579 flags.go:64] FLAG: --logging-format="text"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015976    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015984    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015987    2579 flags.go:64] FLAG: --manifest-url=""
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015990    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015994    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.015997    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016001    2579 flags.go:64] FLAG: --max-pods="110"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016004    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016007    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016010    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016013    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016016    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016018    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016021    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016028    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016031    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016034    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016037    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016040    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016044    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016047    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016049    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016052    2579 flags.go:64] FLAG: --port="10250"
Jul 07 19:09:07.023356 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016055    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016057    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fc3e90ec4e5693ce"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016060    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016063    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016066    2579 flags.go:64] FLAG: --register-node="true"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016068    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016072    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016082    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016085    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016087    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016093    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016097    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016100    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016103    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016105    2579 flags.go:64] FLAG: --runonce="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016108    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016111    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016114    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016117    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016119    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016122    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016125    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016128    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016131    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016134    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 19:09:07.023953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016136    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016139    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016142    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016145    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016147    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016152    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016155    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016158    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016164    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016167    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016170    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016173    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016175    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016178    2579 flags.go:64] FLAG: --v="2"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016182    2579 flags.go:64] FLAG: --version="false"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016191    2579 flags.go:64] FLAG: --vmodule=""
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016195    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.016199    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016336    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016340    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016343    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016346    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016349    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016352    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.024560 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016354    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016357    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016360    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016363    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016365    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016368    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016370    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016373    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016375    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016378    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016380    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016383    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016385    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016388    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016390    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016392    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016395    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016397    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016400    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016402    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.025206 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016405    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016407    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016410    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016412    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016422    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016426    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016431    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016434    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016436    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016439    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016441    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016444    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016446    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016448    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016451    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016454    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016456    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016459    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016461    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016464    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.025698 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016467    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016470    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016472    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016475    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016477    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016480    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016482    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016485    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016487    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016489    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016492    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016494    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016498    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016501    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016504    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016507    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016509    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016515    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016519    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016522    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.026199 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016524    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016526    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016529    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016531    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016534    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016536    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016539    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016541    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016543    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016546    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016548    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016551    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016553    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016556    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016558    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016561    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016565    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016568    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016571    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.026696 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016573    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016576    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016579    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016581    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016584    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016586    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016589    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016591    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016594    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016597    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016602    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016606    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016609    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.016612    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.017167    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 19:09:07.027166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.023944    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.023959    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.024586    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.024594    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024653    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024658    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024661    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024665    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024669    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024671    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024674    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024676    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024679    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024681    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024684    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024687    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024690    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024692    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024695    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024697    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.027551 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024700    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024702    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024705    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024707    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024710    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024712    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024715    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024717    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024719    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024722    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024725    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024727    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024730    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024734    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024736    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024739    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024741    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024744    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024746    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.028070 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024748    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024751    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024753    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024756    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024758    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024761    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024763    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024765    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024768    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024770    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024772    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024775    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024778    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024780    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024782    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024784    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024787    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024790    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024792    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024795    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.028556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024797    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024799    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024803    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024807    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024811    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024815    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024818    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024821    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024824    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024827    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024846    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024850    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024853    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024855    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024858    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024861    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024863    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024866    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024868    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.029108 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024871    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024874    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024876    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024879    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024881    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024884    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024886    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024888    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024891    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024893    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024896    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024899    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024901    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024903    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024906    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024909    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024911    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024914    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024917    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024919    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.029581 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024922    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024925    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024927    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024930    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.024934    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.024939    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025070    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025075    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025078    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025080    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025083    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025085    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025088    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025090    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025092    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 19:09:07.030080 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025095    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025097    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025100    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025102    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025105    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025107    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025110    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025112    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025114    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025117    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025119    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025122    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025124    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025127    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025129    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025132    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025135    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025137    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025139    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025142    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 19:09:07.030442 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025145    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025147    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025149    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025152    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025154    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025157    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025159    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025162    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025165    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025167    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025169    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025172    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025176    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025178    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025181    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025183    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025186    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025188    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025191    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025193    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 19:09:07.030978 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025195    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025198    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025200    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025203    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025205    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025208    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025210    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025213    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025215    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025219    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025223    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025225    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025229    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025231    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025234    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025238    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025241    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025243    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025246    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 19:09:07.031460 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025249    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025252    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025254    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025257    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025259    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025262    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025264    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025267    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025270    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025273    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025275    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025278    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025281    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025283    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025286    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025288    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025290    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025293    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025295    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025298    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 19:09:07.031926 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025301    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025303    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025305    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025308    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025310    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025312    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025315    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025317    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025319    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025322    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:07.025325    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.025329    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.025441    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.027449    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 19:09:07.032404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.028612    2579 server.go:1019] "Starting client certificate rotation"
Jul 07 19:09:07.032740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.028718    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 19:09:07.032740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.028748    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 19:09:07.032740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.030765    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 19:09:07.034610 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.034594    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 19:09:07.044660 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.044640    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 07 19:09:07.048568 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.048546    2579 log.go:25] "Validated CRI v1 image API"
Jul 07 19:09:07.049258 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.049239    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 19:09:07.053234 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.053215    2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 981e5fb1-dd62-4349-88f5-cb230ed816f7:/dev/nvme0n1p4 df099cc4-d41f-4e06-a98c-060fae0cee84:/dev/nvme0n1p3]
Jul 07 19:09:07.053300 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.053234    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 19:09:07.059429 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.059322    2579 manager.go:216] Machine: {Timestamp:2026-07-07 19:09:07.057453737 +0000 UTC m=+0.278659413 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108324 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21063e35fa010192f2328923d59913 SystemUUID:ec21063e-35fa-0101-92f2-328923d59913 BootID:e20e3dc8-c7eb-4601-8c24-69920cd4d792 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0d:8e:44:c8:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0d:8e:44:c8:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:e1:bd:ac:af:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 19:09:07.059429 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.059422    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 19:09:07.059541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.059499    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 19:09:07.060421 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.060396    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 19:09:07.060555 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.060425    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-217.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 19:09:07.061161 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.061146    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 19:09:07.061215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.061163    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 19:09:07.061215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.061170    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 19:09:07.061721 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.061711    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 19:09:07.063007 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.062996    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 19:09:07.064232 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.064217    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 19:09:07.066817 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.066805    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 19:09:07.066948 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.066938    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 19:09:07.066990 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.066968    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 19:09:07.066990 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.066978    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 19:09:07.067053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.066992    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 19:09:07.067053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.067002    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 07 19:09:07.067053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.067010    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 19:09:07.067394 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.067382    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 19:09:07.067437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.067402    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 19:09:07.069504 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.069484    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 19:09:07.073243 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.073219    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 19:09:07.073316 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.073276    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 19:09:07.074370 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074354    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 19:09:07.074370 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074373    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074381    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074389    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074403    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074411    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074422    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074430    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074447    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074455    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074468    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 19:09:07.074553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.074477    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 19:09:07.075149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.075130    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 19:09:07.075149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.075149    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 19:09:07.078547 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.078532    2579 server.go:1268] "Started kubelet"
Jul 07 19:09:07.078620 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.078579    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 19:09:07.078700 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.078629    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 19:09:07.078753 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.078723    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 19:09:07.079358 ip-10-0-136-217 systemd[1]: Started Kubernetes Kubelet.
Jul 07 19:09:07.079711 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.079698    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 19:09:07.080030 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.080015    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 19:09:07.080659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.080640    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-217.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 19:09:07.080659 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.080652    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 19:09:07.080914 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.080877    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 19:09:07.084559 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.084541    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 19:09:07.086113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086099    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 19:09:07.086186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086137    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 19:09:07.086281 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.086266    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.086365 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.085513    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-217.ec2.internal.18c0178d10ce6c10  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-217.ec2.internal,UID:ip-10-0-136-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-217.ec2.internal,},FirstTimestamp:2026-07-07 19:09:07.078507536 +0000 UTC m=+0.299713212,LastTimestamp:2026-07-07 19:09:07.078507536 +0000 UTC m=+0.299713212,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-217.ec2.internal,}"
Jul 07 19:09:07.086742 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086724    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 19:09:07.086742 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086744    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 19:09:07.086920 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086728    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 19:09:07.086920 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086906    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 19:09:07.086920 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.086915    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 19:09:07.087226 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087210    2579 factory.go:55] Registering systemd factory
Jul 07 19:09:07.087286 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087230    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 07 19:09:07.087458 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087445    2579 factory.go:153] Registering CRI-O factory
Jul 07 19:09:07.087509 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087461    2579 factory.go:223] Registration of the crio container factory successfully
Jul 07 19:09:07.087509 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087507    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 19:09:07.087599 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087527    2579 factory.go:103] Registering Raw factory
Jul 07 19:09:07.087599 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.087541    2579 manager.go:1196] Started watching for new ooms in manager
Jul 07 19:09:07.088239 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.088223    2579 manager.go:349] Starting recovery of all containers
Jul 07 19:09:07.089906 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.089883    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 19:09:07.090002 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.089974    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 19:09:07.100785 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.098821    2579 manager.go:354] Recovery completed
Jul 07 19:09:07.104420 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.104406    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.106146 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106130    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.106208 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106160    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.106208 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106173    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.106721 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106707    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 19:09:07.106770 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106723    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 19:09:07.106770 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106743    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 19:09:07.106770 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.106714    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.107428 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.107412    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.107491 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.107438    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.107491 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.107448    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.108738 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.108676    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-217.ec2.internal.18c0178d12742a1b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-217.ec2.internal,UID:ip-10-0-136-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-217.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-217.ec2.internal,},FirstTimestamp:2026-07-07 19:09:07.106146843 +0000 UTC m=+0.327352522,LastTimestamp:2026-07-07 19:09:07.106146843 +0000 UTC m=+0.327352522,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-217.ec2.internal,}"
Jul 07 19:09:07.108998 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.108984    2579 policy_none.go:49] "None policy: Start"
Jul 07 19:09:07.109039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.109006    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 19:09:07.109039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.109021    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 19:09:07.111011 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.111000    2579 policy_none.go:47] "Start"
Jul 07 19:09:07.118464 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.118405    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-217.ec2.internal.18c0178d12747066  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-217.ec2.internal,UID:ip-10-0-136-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-217.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-217.ec2.internal,},FirstTimestamp:2026-07-07 19:09:07.106164838 +0000 UTC m=+0.327370515,LastTimestamp:2026-07-07 19:09:07.106164838 +0000 UTC m=+0.327370515,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-217.ec2.internal,}"
Jul 07 19:09:07.118555 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.118540    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-th8m4"
Jul 07 19:09:07.127594 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.127572    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-th8m4"
Jul 07 19:09:07.130920 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.130804    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-217.ec2.internal.18c0178d1274aa0c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-217.ec2.internal,UID:ip-10-0-136-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-217.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-217.ec2.internal,},FirstTimestamp:2026-07-07 19:09:07.106179596 +0000 UTC m=+0.327385272,LastTimestamp:2026-07-07 19:09:07.106179596 +0000 UTC m=+0.327385272,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-217.ec2.internal,}"
Jul 07 19:09:07.144426 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144408    2579 manager.go:337] "Starting Device Plugin manager"
Jul 07 19:09:07.144489 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.144443    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 19:09:07.144489 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144453    2579 server.go:87] "Starting device plugin registration server"
Jul 07 19:09:07.144694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144680    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 19:09:07.144755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144697    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 19:09:07.144816 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144759    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 19:09:07.144882 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144826    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 19:09:07.144882 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.144854    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 19:09:07.146335 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.146311    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 19:09:07.146409 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.146372    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.245656 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.245638    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.247760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.246412    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.247760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.246443    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.247760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.246456    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.247760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.246485    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.254400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.254387    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.254457 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.254408    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-217.ec2.internal\": node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.280444 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.280378    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 19:09:07.281719 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.281703    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 19:09:07.281790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.281756    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 19:09:07.281790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.281781    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 19:09:07.281907 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.281820    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 19:09:07.284206 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.284188    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 19:09:07.285093 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.285072    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.382153 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.382097    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"]
Jul 07 19:09:07.382313 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.382198    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.383114 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.383098    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.383180 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.383127    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.383180 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.383139    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.385250 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.385237    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.385418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.385407    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.385527 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.385515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.385572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.385549    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.386119 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386088    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.386119 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386094    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.386264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386124    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.386264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386144    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.386264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386157    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.386264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.386179    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.388479 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.388462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.388533 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.388498    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:07.388624 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.388605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.388676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.388641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.388676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.388663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0242a47c8f42188a150225d6e13e270-config\") pod \"kube-apiserver-proxy-ip-10-0-136-217.ec2.internal\" (UID: \"e0242a47c8f42188a150225d6e13e270\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.389167 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.389141    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:07.389250 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.389175    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:07.389250 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.389189    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:07.419566 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.419546    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-217.ec2.internal\" not found" node="ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.423922 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.423907    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-217.ec2.internal\" not found" node="ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.485362 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.485327    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.489571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0242a47c8f42188a150225d6e13e270-config\") pod \"kube-apiserver-proxy-ip-10-0-136-217.ec2.internal\" (UID: \"e0242a47c8f42188a150225d6e13e270\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.489663 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.489663 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.489663 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489643    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.489663 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0242a47c8f42188a150225d6e13e270-config\") pod \"kube-apiserver-proxy-ip-10-0-136-217.ec2.internal\" (UID: \"e0242a47c8f42188a150225d6e13e270\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.489663 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.489652    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3cfd0e1ebd8169b57d9e712a094a397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal\" (UID: \"d3cfd0e1ebd8169b57d9e712a094a397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.586268 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.586219    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.686692 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.686653    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:07.722115 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.722077    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.727158 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:07.726997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:07.787259 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:07.787225    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.147503 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.147440    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.247857 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.247836    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.288143 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.288126    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 19:09:10.288621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.288246    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 19:09:10.348020 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.347996    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.391971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.391953    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 19:09:10.392676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.392640    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 19:04:09 +0000 UTC" deadline="2028-01-05 02:54:44.84036639 +0000 UTC"
Jul 07 19:09:10.392676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.392675    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13111h45m34.447694695s"
Jul 07 19:09:10.392830 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.392807    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 19:09:10.392871 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.392834    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 19:09:10.392871 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.392845    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeHasSufficientPID"
Jul 07 19:09:10.448530 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.448480    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.549010 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.548984    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.572574 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:10.572534    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0242a47c8f42188a150225d6e13e270.slice/crio-911d4f51696afd350b0c0d55196ea46f40436845108f5ab1b7252803dad6af27 WatchSource:0}: Error finding container 911d4f51696afd350b0c0d55196ea46f40436845108f5ab1b7252803dad6af27: Status 404 returned error can't find the container with id 911d4f51696afd350b0c0d55196ea46f40436845108f5ab1b7252803dad6af27
Jul 07 19:09:10.572988 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:10.572967    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cfd0e1ebd8169b57d9e712a094a397.slice/crio-e9610ae2f93dca6c7e073e28fc93aee24e77400acdae30fa150fb79db865185b WatchSource:0}: Error finding container e9610ae2f93dca6c7e073e28fc93aee24e77400acdae30fa150fb79db865185b: Status 404 returned error can't find the container with id e9610ae2f93dca6c7e073e28fc93aee24e77400acdae30fa150fb79db865185b
Jul 07 19:09:10.577108 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.577094    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 19:09:10.649695 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:10.649658    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-217.ec2.internal\" not found"
Jul 07 19:09:10.649992 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.649976    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 19:09:10.746572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.746502    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:10.757621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.757598    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 19:09:10.758996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.758984    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal"
Jul 07 19:09:10.771761 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.771742    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 19:09:10.941766 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.941738    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 19:09:10.948402 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:10.948383    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 19:09:11.327706 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.327680    2579 apiserver.go:52] "Watching apiserver"
Jul 07 19:09:11.332270 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.332245    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 19:09:11.332638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.332611    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-82trz","kube-system/global-pull-secret-syncer-wp8nm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2","openshift-cluster-node-tuning-operator/tuned-p2qpp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal","openshift-multus/multus-additional-cni-plugins-hb76f","openshift-multus/network-metrics-daemon-5xrb6","openshift-network-diagnostics/network-check-target-n2tls","openshift-network-operator/iptables-alerter-djljf","kube-system/konnectivity-agent-v9nhr","kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal","openshift-dns/node-resolver-l69s9","openshift-image-registry/node-ca-85qcf","openshift-multus/multus-kqvd4"]
Jul 07 19:09:11.335110 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.335091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.335210 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.335176    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:11.337222 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.337201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:11.337309 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.337280    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:11.339851 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.339824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.341755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.341714    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 19:09:11.341755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.341738    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7pgc4\""
Jul 07 19:09:11.341755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.341750    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.341974 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.341960    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.342418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.342399    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.343735 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.343706    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mz47m\""
Jul 07 19:09:11.343735 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.343720    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.343845 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.343719    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.344269 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.344256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.345807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.345787    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lcxr6\""
Jul 07 19:09:11.345907 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.345805    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 19:09:11.345907 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.345805    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 19:09:11.345907 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.345805    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.346146 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.346133    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 19:09:11.346267 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.346253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 19:09:11.346328 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.346314    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.346523 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.346508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.346649 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.346630    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 19:09:11.348092 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348076    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 19:09:11.348184 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348120    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.348340 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348320    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 19:09:11.348422 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348372    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-59nvl\""
Jul 07 19:09:11.348422 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348400    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.348628 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348609    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 19:09:11.348916 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.348900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.350435 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.350404    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nqknm\""
Jul 07 19:09:11.350786 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.350768    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.350891 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.350828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 19:09:11.350891 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.350851    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.351175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.351152    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.353694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.352973    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 19:09:11.353694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.353123    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.353694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.353128    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5tg95\""
Jul 07 19:09:11.353694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.353620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.353694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.353643    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.355252 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.354123    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:11.355448 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355395    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-var-lib-kubelet\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355448 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-host\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355540 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-tuned\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355540 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-tmp\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355540 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-env-overrides\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.355695 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-run\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355695 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-node-log\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.355695 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-cnibin\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.355695 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-socket-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-systemd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-systemd\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-etc-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.355859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-bin\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovn-node-metrics-cert\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-system-cni-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.355973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-registration-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-sys-fs\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhkcp\" (UniqueName: \"kubernetes.io/projected/41c18592-cc5f-4fdc-ac17-f957986f0b3f-kube-api-access-qhkcp\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-var-lib-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356111 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zv92\" (UniqueName: \"kubernetes.io/projected/dc8237c9-6a76-49c4-b86c-818f95b3aa82-kube-api-access-8zv92\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-kubernetes\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-lib-modules\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-ovn\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-kubelet\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-netns\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-log-socket\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-device-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.356544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvp5\" (UniqueName: \"kubernetes.io/projected/9832adb6-fada-441b-b0ce-afadc0758567-kube-api-access-lfvp5\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-modprobe-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-conf\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-config\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-sys\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.357000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.356984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysconfig\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sds2z\" (UniqueName: \"kubernetes.io/projected/02c48293-e955-4f10-ad91-2ea8c1a314af-kube-api-access-sds2z\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-slash\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-systemd-units\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-netd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjtqz\" (UniqueName: \"kubernetes.io/projected/3781fe31-12a2-450f-9331-ff8740b14849-kube-api-access-pjtqz\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357178    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41c18592-cc5f-4fdc-ac17-f957986f0b3f-iptables-alerter-script\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.357385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357420    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-script-lib\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdctl\" (UniqueName: \"kubernetes.io/projected/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-kube-api-access-jdctl\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-os-release\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.357705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.357575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41c18592-cc5f-4fdc-ac17-f957986f0b3f-host-slash\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.358889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.358622    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 19:09:11.358889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.358733    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 19:09:11.358889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.358752    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qznx5\""
Jul 07 19:09:11.359750 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.359688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.360083 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.360044    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 19:09:11.361447 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.361429    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 19:09:11.361675 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.361657    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xd98w\""
Jul 07 19:09:11.361999 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.361978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.363525 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.363504    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 19:09:11.363666 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.363652    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 19:09:11.363842 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.363827    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-plnz9\""
Jul 07 19:09:11.445897 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.445873    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wr9s5"
Jul 07 19:09:11.447581 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.447562    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 19:09:11.455331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.455310    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wr9s5"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed443871-0f3a-487a-afe6-38fe7fc9ae81-serviceca\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-multus-certs\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.457990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-script-lib\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458004    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458013    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.458054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdctl\" (UniqueName: \"kubernetes.io/projected/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-kube-api-access-jdctl\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-os-release\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41c18592-cc5f-4fdc-ac17-f957986f0b3f-host-slash\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458311    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-os-release\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qhnz\" (UniqueName: \"kubernetes.io/projected/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-kube-api-access-4qhnz\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458347    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41c18592-cc5f-4fdc-ac17-f957986f0b3f-host-slash\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-var-lib-kubelet\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-host\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458421    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-var-lib-kubelet\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-tuned\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458487    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-host\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.458548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-tmp\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-env-overrides\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458574    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-script-lib\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-multus\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-run\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-node-log\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-tmp-dir\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-kubelet-config\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-agent-certs\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-k8s-cni-cncf-io\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-etc-kubernetes\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-cnibin\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458836    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-node-log\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458842    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458857    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-socket-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458956    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-cnibin\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.459147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458961    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-env-overrides\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.458995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-dbus\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-os-release\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-systemd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-systemd\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459128    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-socket-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459162    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-etc-selinux\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-konnectivity-ca\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-etc-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459220    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-systemd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459274    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-systemd\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459288    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-bin\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovn-node-metrics-cert\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459326    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-run\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.460021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-system-cni-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459374    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-etc-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459399    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-bin\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-registration-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459438    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3781fe31-12a2-450f-9331-ff8740b14849-system-cni-dir\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-sys-fs\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459490    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-registration-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459491    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhkcp\" (UniqueName: \"kubernetes.io/projected/41c18592-cc5f-4fdc-ac17-f957986f0b3f-kube-api-access-qhkcp\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459528    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-sys-fs\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-var-lib-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed443871-0f3a-487a-afe6-38fe7fc9ae81-host\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.459857    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459855    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-var-lib-openvswitch\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.459921    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:11.959902106 +0000 UTC m=+2.921353366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:11.460971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-netns\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.459983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-kubelet\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-hostroot\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zv92\" (UniqueName: \"kubernetes.io/projected/dc8237c9-6a76-49c4-b86c-818f95b3aa82-kube-api-access-8zv92\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-kubernetes\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-hosts-file\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-lib-modules\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-ovn\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460167    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-kubernetes\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-conf-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-daemon-config\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vhm2\" (UniqueName: \"kubernetes.io/projected/77a579fd-6f96-4df7-8fbd-085b232bda8a-kube-api-access-4vhm2\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-kubelet\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-netns\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.461772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460376    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-log-socket\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-device-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-socket-dir-parent\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460501    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-lib-modules\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvp5\" (UniqueName: \"kubernetes.io/projected/9832adb6-fada-441b-b0ce-afadc0758567-kube-api-access-lfvp5\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-run-ovn\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460603    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-kubelet\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460649    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-log-socket\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460700    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-run-netns\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460729    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-modprobe-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460780    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9832adb6-fada-441b-b0ce-afadc0758567-device-dir\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.460999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-conf\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-config\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.462494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-sys\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461118    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysctl-conf\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysconfig\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461167    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-sysconfig\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sds2z\" (UniqueName: \"kubernetes.io/projected/02c48293-e955-4f10-ad91-2ea8c1a314af-kube-api-access-sds2z\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-slash\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9hk5\" (UniqueName: \"kubernetes.io/projected/ed443871-0f3a-487a-afe6-38fe7fc9ae81-kube-api-access-w9hk5\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-systemd-units\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-netd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-systemd-units\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-system-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-cnibin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-cni-binary-copy\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-bin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtqz\" (UniqueName: \"kubernetes.io/projected/3781fe31-12a2-450f-9331-ff8740b14849-kube-api-access-pjtqz\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41c18592-cc5f-4fdc-ac17-f957986f0b3f-iptables-alerter-script\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.463175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461720    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.461809    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-modprobe-d\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462187    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41c18592-cc5f-4fdc-ac17-f957986f0b3f-iptables-alerter-script\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462235    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-cni-netd\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462278    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02c48293-e955-4f10-ad91-2ea8c1a314af-sys\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462281    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3781fe31-12a2-450f-9331-ff8740b14849-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462476    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-host-slash\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462541    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-etc-tuned\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462721    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovnkube-config\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.462920    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02c48293-e955-4f10-ad91-2ea8c1a314af-tmp\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.463759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.463220    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-ovn-node-metrics-cert\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.466661 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.466641    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdctl\" (UniqueName: \"kubernetes.io/projected/b6cba104-1d0b-4caa-ae20-ee37d44d6fdb-kube-api-access-jdctl\") pod \"ovnkube-node-82trz\" (UID: \"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.467003 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.466912    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:11.467003 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.466933    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:11.467003 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.466945    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:11.467003 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.466994    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:11.966978093 +0000 UTC m=+2.928429351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:11.468336 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.468310    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhkcp\" (UniqueName: \"kubernetes.io/projected/41c18592-cc5f-4fdc-ac17-f957986f0b3f-kube-api-access-qhkcp\") pod \"iptables-alerter-djljf\" (UID: \"41c18592-cc5f-4fdc-ac17-f957986f0b3f\") " pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.469769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.469749    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zv92\" (UniqueName: \"kubernetes.io/projected/dc8237c9-6a76-49c4-b86c-818f95b3aa82-kube-api-access-8zv92\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.470545 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.470525    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjtqz\" (UniqueName: \"kubernetes.io/projected/3781fe31-12a2-450f-9331-ff8740b14849-kube-api-access-pjtqz\") pod \"multus-additional-cni-plugins-hb76f\" (UID: \"3781fe31-12a2-450f-9331-ff8740b14849\") " pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.471184 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.471163    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvp5\" (UniqueName: \"kubernetes.io/projected/9832adb6-fada-441b-b0ce-afadc0758567-kube-api-access-lfvp5\") pod \"aws-ebs-csi-driver-node-sqqx2\" (UID: \"9832adb6-fada-441b-b0ce-afadc0758567\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.471407 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.471388    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sds2z\" (UniqueName: \"kubernetes.io/projected/02c48293-e955-4f10-ad91-2ea8c1a314af-kube-api-access-sds2z\") pod \"tuned-p2qpp\" (UID: \"02c48293-e955-4f10-ad91-2ea8c1a314af\") " pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.546146 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.546087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal" event={"ID":"d3cfd0e1ebd8169b57d9e712a094a397","Type":"ContainerStarted","Data":"e9610ae2f93dca6c7e073e28fc93aee24e77400acdae30fa150fb79db865185b"}
Jul 07 19:09:11.547306 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.547281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal" event={"ID":"e0242a47c8f42188a150225d6e13e270","Type":"ContainerStarted","Data":"911d4f51696afd350b0c0d55196ea46f40436845108f5ab1b7252803dad6af27"}
Jul 07 19:09:11.562521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-hosts-file\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.562643 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.562643 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-conf-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562643 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-daemon-config\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562643 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vhm2\" (UniqueName: \"kubernetes.io/projected/77a579fd-6f96-4df7-8fbd-085b232bda8a-kube-api-access-4vhm2\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562643 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-socket-dir-parent\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562658    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-conf-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562679    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-socket-dir-parent\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562706    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-hosts-file\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9hk5\" (UniqueName: \"kubernetes.io/projected/ed443871-0f3a-487a-afe6-38fe7fc9ae81-kube-api-access-w9hk5\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-system-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-cnibin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.562866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-cni-binary-copy\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-bin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed443871-0f3a-487a-afe6-38fe7fc9ae81-serviceca\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.562931    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-multus-certs\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.562976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qhnz\" (UniqueName: \"kubernetes.io/projected/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-kube-api-access-4qhnz\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.562989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:12.062971719 +0000 UTC m=+3.024422973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-multus\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-tmp-dir\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-kubelet-config\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-agent-certs\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-k8s-cni-cncf-io\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-etc-kubernetes\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-dbus\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-os-release\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-konnectivity-ca\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563261    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-kubelet-config\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563291    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-system-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563330    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-cnibin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563344 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563331    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-multus\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563619    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-tmp-dir\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.563790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563709    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-cni-binary-copy\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563814    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-dbus\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:11.563859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed443871-0f3a-487a-afe6-38fe7fc9ae81-host\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.563859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563850    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-multus-certs\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-netns\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563871    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-k8s-cni-cncf-io\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-kubelet\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563911    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-cni-dir\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-hostroot\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.563951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.563215    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77a579fd-6f96-4df7-8fbd-085b232bda8a-multus-daemon-config\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564033    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-hostroot\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564053    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed443871-0f3a-487a-afe6-38fe7fc9ae81-serviceca\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564065    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed443871-0f3a-487a-afe6-38fe7fc9ae81-host\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564089    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-etc-kubernetes\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564107    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-kubelet\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564129 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564119    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-os-release\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564162    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-var-lib-cni-bin\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564189    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77a579fd-6f96-4df7-8fbd-085b232bda8a-host-run-netns\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.564399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.564352    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-konnectivity-ca\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.566185 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.566160    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be1de55-29fc-40d0-a4a7-21a2b9a95c6a-agent-certs\") pod \"konnectivity-agent-v9nhr\" (UID: \"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a\") " pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.576437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.576412    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vhm2\" (UniqueName: \"kubernetes.io/projected/77a579fd-6f96-4df7-8fbd-085b232bda8a-kube-api-access-4vhm2\") pod \"multus-kqvd4\" (UID: \"77a579fd-6f96-4df7-8fbd-085b232bda8a\") " pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.578114 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.578059    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9hk5\" (UniqueName: \"kubernetes.io/projected/ed443871-0f3a-487a-afe6-38fe7fc9ae81-kube-api-access-w9hk5\") pod \"node-ca-85qcf\" (UID: \"ed443871-0f3a-487a-afe6-38fe7fc9ae81\") " pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.580817 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.580797    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qhnz\" (UniqueName: \"kubernetes.io/projected/5efe5858-ff97-4429-b89f-85e7cc3cf1a4-kube-api-access-4qhnz\") pod \"node-resolver-l69s9\" (UID: \"5efe5858-ff97-4429-b89f-85e7cc3cf1a4\") " pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.650604 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.650563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2"
Jul 07 19:09:11.661580 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.661547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p2qpp"
Jul 07 19:09:11.661918 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.661894    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9832adb6_fada_441b_b0ce_afadc0758567.slice/crio-5e1f2a81b2b344076bcf507f6c9c9416adb661d2f37be8f1b9a2806e1a0369c1 WatchSource:0}: Error finding container 5e1f2a81b2b344076bcf507f6c9c9416adb661d2f37be8f1b9a2806e1a0369c1: Status 404 returned error can't find the container with id 5e1f2a81b2b344076bcf507f6c9c9416adb661d2f37be8f1b9a2806e1a0369c1
Jul 07 19:09:11.669824 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.669807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:11.670143 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.670117    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c48293_e955_4f10_ad91_2ea8c1a314af.slice/crio-4652fd944d8635c7e2849e49ff91537591315d722be6d0b11068a64bb66656f3 WatchSource:0}: Error finding container 4652fd944d8635c7e2849e49ff91537591315d722be6d0b11068a64bb66656f3: Status 404 returned error can't find the container with id 4652fd944d8635c7e2849e49ff91537591315d722be6d0b11068a64bb66656f3
Jul 07 19:09:11.674672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.674655    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hb76f"
Jul 07 19:09:11.678694 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.678666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6cba104_1d0b_4caa_ae20_ee37d44d6fdb.slice/crio-b9c4fe9bfaaa68d551a08aad214e1fb0f0ce61394a61ae39efc2abcd0afa6efa WatchSource:0}: Error finding container b9c4fe9bfaaa68d551a08aad214e1fb0f0ce61394a61ae39efc2abcd0afa6efa: Status 404 returned error can't find the container with id b9c4fe9bfaaa68d551a08aad214e1fb0f0ce61394a61ae39efc2abcd0afa6efa
Jul 07 19:09:11.683667 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.683649    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-djljf"
Jul 07 19:09:11.684236 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.684217    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3781fe31_12a2_450f_9331_ff8740b14849.slice/crio-a2b14abeee3683048375924173ece6f244c863fa124026976fb54f693723ecf2 WatchSource:0}: Error finding container a2b14abeee3683048375924173ece6f244c863fa124026976fb54f693723ecf2: Status 404 returned error can't find the container with id a2b14abeee3683048375924173ece6f244c863fa124026976fb54f693723ecf2
Jul 07 19:09:11.691369 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.691349    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-85qcf"
Jul 07 19:09:11.693697 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.693677    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c18592_cc5f_4fdc_ac17_f957986f0b3f.slice/crio-01bb98b98070d8714bbb867a117aec6e4ef31446808599c1350d429633ed5cf9 WatchSource:0}: Error finding container 01bb98b98070d8714bbb867a117aec6e4ef31446808599c1350d429633ed5cf9: Status 404 returned error can't find the container with id 01bb98b98070d8714bbb867a117aec6e4ef31446808599c1350d429633ed5cf9
Jul 07 19:09:11.696888 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.696866    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:11.700325 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.700307    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded443871_0f3a_487a_afe6_38fe7fc9ae81.slice/crio-249a54606265816653b2ff61891b3082ec6a2820ef62609bdcf972da0a91525a WatchSource:0}: Error finding container 249a54606265816653b2ff61891b3082ec6a2820ef62609bdcf972da0a91525a: Status 404 returned error can't find the container with id 249a54606265816653b2ff61891b3082ec6a2820ef62609bdcf972da0a91525a
Jul 07 19:09:11.702989 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.702850    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqvd4"
Jul 07 19:09:11.706186 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.706166    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be1de55_29fc_40d0_a4a7_21a2b9a95c6a.slice/crio-d6690fbc9144e2b4727e7d256df1871c098c0e49551cf68e20337bacee5225e2 WatchSource:0}: Error finding container d6690fbc9144e2b4727e7d256df1871c098c0e49551cf68e20337bacee5225e2: Status 404 returned error can't find the container with id d6690fbc9144e2b4727e7d256df1871c098c0e49551cf68e20337bacee5225e2
Jul 07 19:09:11.708329 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.708247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l69s9"
Jul 07 19:09:11.713497 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.713473    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a579fd_6f96_4df7_8fbd_085b232bda8a.slice/crio-e6dff1179a3ed4f9bf5dd1ea8cfa97bb866b82028b86540ed6962c8f330ab44d WatchSource:0}: Error finding container e6dff1179a3ed4f9bf5dd1ea8cfa97bb866b82028b86540ed6962c8f330ab44d: Status 404 returned error can't find the container with id e6dff1179a3ed4f9bf5dd1ea8cfa97bb866b82028b86540ed6962c8f330ab44d
Jul 07 19:09:11.717782 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:11.717761    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efe5858_ff97_4429_b89f_85e7cc3cf1a4.slice/crio-244e9d3a8a322767fc30c27e95f3f8c3504695017ff0aab46c970bfb94e0ffb8 WatchSource:0}: Error finding container 244e9d3a8a322767fc30c27e95f3f8c3504695017ff0aab46c970bfb94e0ffb8: Status 404 returned error can't find the container with id 244e9d3a8a322767fc30c27e95f3f8c3504695017ff0aab46c970bfb94e0ffb8
Jul 07 19:09:11.742565 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.742546    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 19:09:11.966946 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:11.966864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:11.967083 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.967018    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:11.967083 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:11.967082    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:12.967068632 +0000 UTC m=+3.928519891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:12.067417 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.067371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:12.067573 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.067425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:12.067573 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067537    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:12.067573 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067565    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:12.067736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067581    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:12.067736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067614    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:12.067736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067623    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:13.067604994 +0000 UTC m=+4.029056270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:12.067736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.067650    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:13.067639001 +0000 UTC m=+4.029090269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:12.458506 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.458461    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:04:11 +0000 UTC" deadline="2028-01-22 19:23:44.267026802 +0000 UTC"
Jul 07 19:09:12.458506 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.458501    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13536h14m31.808530471s"
Jul 07 19:09:12.542957 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.542430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:12.542957 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.542561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:12.555609 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.555562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"b9c4fe9bfaaa68d551a08aad214e1fb0f0ce61394a61ae39efc2abcd0afa6efa"}
Jul 07 19:09:12.562818 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.562789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l69s9" event={"ID":"5efe5858-ff97-4429-b89f-85e7cc3cf1a4","Type":"ContainerStarted","Data":"244e9d3a8a322767fc30c27e95f3f8c3504695017ff0aab46c970bfb94e0ffb8"}
Jul 07 19:09:12.568281 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.568224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-85qcf" event={"ID":"ed443871-0f3a-487a-afe6-38fe7fc9ae81","Type":"ContainerStarted","Data":"249a54606265816653b2ff61891b3082ec6a2820ef62609bdcf972da0a91525a"}
Jul 07 19:09:12.573510 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.573485    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-djljf" event={"ID":"41c18592-cc5f-4fdc-ac17-f957986f0b3f","Type":"ContainerStarted","Data":"01bb98b98070d8714bbb867a117aec6e4ef31446808599c1350d429633ed5cf9"}
Jul 07 19:09:12.575102 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.575068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerStarted","Data":"a2b14abeee3683048375924173ece6f244c863fa124026976fb54f693723ecf2"}
Jul 07 19:09:12.577260 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.577233    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p2qpp" event={"ID":"02c48293-e955-4f10-ad91-2ea8c1a314af","Type":"ContainerStarted","Data":"4652fd944d8635c7e2849e49ff91537591315d722be6d0b11068a64bb66656f3"}
Jul 07 19:09:12.581667 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.581642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2" event={"ID":"9832adb6-fada-441b-b0ce-afadc0758567","Type":"ContainerStarted","Data":"5e1f2a81b2b344076bcf507f6c9c9416adb661d2f37be8f1b9a2806e1a0369c1"}
Jul 07 19:09:12.587520 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.587493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqvd4" event={"ID":"77a579fd-6f96-4df7-8fbd-085b232bda8a","Type":"ContainerStarted","Data":"e6dff1179a3ed4f9bf5dd1ea8cfa97bb866b82028b86540ed6962c8f330ab44d"}
Jul 07 19:09:12.590861 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.590836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9nhr" event={"ID":"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a","Type":"ContainerStarted","Data":"d6690fbc9144e2b4727e7d256df1871c098c0e49551cf68e20337bacee5225e2"}
Jul 07 19:09:12.975879 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:12.975657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:12.976154 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.976128    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:12.976226 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:12.976193    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:14.976175149 +0000 UTC m=+5.937626419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:13.076862 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.076823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:13.077046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.076879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:13.077046 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077025    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:13.077046 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077042    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:13.077209 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077055    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:13.077209 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077115    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:15.077096799 +0000 UTC m=+6.038548056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:13.077656 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077509    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:13.077656 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.077562    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:15.077547653 +0000 UTC m=+6.038998913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:13.459003 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.458932    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:04:11 +0000 UTC" deadline="2028-01-26 17:18:23.630571033 +0000 UTC"
Jul 07 19:09:13.459003 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.458993    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13630h9m10.171581879s"
Jul 07 19:09:13.545601 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.543463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:13.545601 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.543609    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:13.545601 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.545403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:13.545601 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:13.545522    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:13.607361 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:13.607318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal" event={"ID":"e0242a47c8f42188a150225d6e13e270","Type":"ContainerStarted","Data":"1798eb57c6b091df9d1162cecae883ca2079e817acad6d3525b4cd767307a112"}
Jul 07 19:09:14.542564 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:14.542527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:14.543063 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:14.542674    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:14.994377 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:14.993731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:14.994377 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:14.993907    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:14.994377 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:14.993968    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:18.993949898 +0000 UTC m=+9.955401170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:15.094463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:15.094520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.094690    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.094713    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.094727    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.094786    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:19.094768553 +0000 UTC m=+10.056219810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.095188    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:15.095301 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.095238    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:19.095223158 +0000 UTC m=+10.056674428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:15.544717 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:15.544079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:15.544717 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.544203    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:15.544717 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:15.544554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:15.544717 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:15.544682    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:16.542352 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:16.542317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:16.542537 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:16.542451    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:17.542578 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:17.542046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:17.542578 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:17.542082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:17.542578 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:17.542193    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:17.542578 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:17.542354    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:18.542571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:18.542531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:18.542775 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:18.542670    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:19.027777 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:19.027747    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:19.027932 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.027892    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:19.027998 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.027975    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:27.027954073 +0000 UTC m=+17.989405330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:19.128365 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:19.128326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:19.128377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128455    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128492    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128511    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128524    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:19.128544 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128527    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:27.128504809 +0000 UTC m=+18.089956085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:19.128892 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.128576    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:27.1285581 +0000 UTC m=+18.090009374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:19.543112 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:19.542853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:19.543112 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.542936    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:19.543556 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:19.543244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:19.543556 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:19.543331    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:20.542127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:20.542091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:20.542295 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:20.542207    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:21.544632 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:21.544583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:21.545072 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:21.544713    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:21.545116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:21.545087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:21.545212 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:21.545164    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:22.542017 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:22.541979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:22.542171 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:22.542085    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:23.542973 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:23.542448    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:23.542973 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:23.542608    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:23.542973 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:23.542628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:23.542973 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:23.542739    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:24.542100 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:24.542062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:24.542274 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:24.542182    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:25.541808 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:25.541780    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:25.542274 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:25.541776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:25.542274 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:25.541922    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:25.542274 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:25.541980    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:26.542314 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:26.542284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:26.542755 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:26.542395    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:27.088244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:27.088191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:27.088425 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.088354    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:27.088425 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.088420    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:43.088405213 +0000 UTC m=+34.049856468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:27.188847 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:27.188811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:27.189004 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:27.188870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:27.189004 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.188963    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:27.189125 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.189026    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:27.189125 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.189053    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:27.189125 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.189066    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:27.189125 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.189033    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:43.189014624 +0000 UTC m=+34.150465902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:27.189291 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.189131    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:09:43.189114194 +0000 UTC m=+34.150565459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:27.541858 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:27.541823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:27.542046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:27.541869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:27.542046 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.541938    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:27.542155 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:27.542101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:28.541779 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:28.541741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:28.542158 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:28.541863    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:29.542392 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:29.542355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:29.542813 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:29.542469    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:29.542813 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:29.542524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:29.542813 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:29.542660    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:30.542128 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:30.542099    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:30.542311 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:30.542215    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:31.542117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.542090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:31.542413 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:31.542216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:31.542413 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.542253    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:31.542481 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:31.542402    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:31.649232 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.649067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"62826cfe2877fe02dcfe0a87cfb3acb4bc6c7297759872c707ae9623df6fbfa7"}
Jul 07 19:09:31.649347 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.649275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"dc8582f6a4063ab38f467fcfd059fd807d241b43af754b6a7ecb767b2f45e0d8"}
Jul 07 19:09:31.650738 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.650710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p2qpp" event={"ID":"02c48293-e955-4f10-ad91-2ea8c1a314af","Type":"ContainerStarted","Data":"d0620ca0d1ab0d2177ec8edebfff5196adaabe2f6922511b5b6ea9278d6f7da9"}
Jul 07 19:09:31.652009 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.651985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqvd4" event={"ID":"77a579fd-6f96-4df7-8fbd-085b232bda8a","Type":"ContainerStarted","Data":"7a1bf063e9259817a603ce4862d92fec9e98fa9058fa8238aa486be2bb1fafb8"}
Jul 07 19:09:31.682262 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.682213    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p2qpp" podStartSLOduration=2.973218876 podStartE2EDuration="22.682200301s" totalImagesPullingTime="19.708981425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:31.681906551 +0000 UTC m=+22.643357823" watchObservedRunningTime="2026-07-07 19:09:31.682200301 +0000 UTC m=+22.643651583"
Jul 07 19:09:31.682553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.682531    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-217.ec2.internal" podStartSLOduration=21.682524122 podStartE2EDuration="21.682524122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:13.62420585 +0000 UTC m=+4.585657130" watchObservedRunningTime="2026-07-07 19:09:31.682524122 +0000 UTC m=+22.643975399"
Jul 07 19:09:31.709507 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:31.709467    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kqvd4" podStartSLOduration=2.994164407 podStartE2EDuration="22.709450219s" totalImagesPullingTime="19.715285812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:31.709366821 +0000 UTC m=+22.670818099" watchObservedRunningTime="2026-07-07 19:09:31.709450219 +0000 UTC m=+22.670901496"
Jul 07 19:09:32.542561 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.542526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:32.543355 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:32.542658    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:32.655444 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.655413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9nhr" event={"ID":"7be1de55-29fc-40d0-a4a7-21a2b9a95c6a","Type":"ContainerStarted","Data":"c6e0d3a197b9b59771e4646da85d3b717c12c54cb13506ce88acbfe1924df7d8"}
Jul 07 19:09:32.658242 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.658219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"3ae68b340027799bcdd54e7fdf7f44d9a563886b6c348ee7992b0c83169daae3"}
Jul 07 19:09:32.658384 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.658289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"e70b5374f95d4d64820ebeb08c35958ba35d395e3b43a2b0bce8fcbe085d4a80"}
Jul 07 19:09:32.658384 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.658320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"609ff6352d6e1865ba404876013ca79b534a0d1f9192f9f4faf9c9a8beca0ae4"}
Jul 07 19:09:32.658384 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.658362    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"54573e79151a900d834745b1a87095afa1759e4ea9523f32989e867310fc5e4d"}
Jul 07 19:09:32.659733 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.659707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l69s9" event={"ID":"5efe5858-ff97-4429-b89f-85e7cc3cf1a4","Type":"ContainerStarted","Data":"e6c73d54a492363880c6b423c5ce332f39fc316c37cb37de8cb27f084c0d45e8"}
Jul 07 19:09:32.661264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.661238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-85qcf" event={"ID":"ed443871-0f3a-487a-afe6-38fe7fc9ae81","Type":"ContainerStarted","Data":"bce73c972db8f3c8c60bc9f3ed3bbebcf6176d4ee9a96ca6bbdb599e6dc4ad36"}
Jul 07 19:09:32.662552 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.662531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-djljf" event={"ID":"41c18592-cc5f-4fdc-ac17-f957986f0b3f","Type":"ContainerStarted","Data":"5a7d924ca3d794da048723f8cc02e79d82b7fcaeebdd7efdfa3fbe57894a4092"}
Jul 07 19:09:32.664085 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.663861    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="6bc5a70f3f418c1dcbf5df815af857e9338e6f538eb25fa68d65485d63d79eda" exitCode=0
Jul 07 19:09:32.664171 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.663936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"6bc5a70f3f418c1dcbf5df815af857e9338e6f538eb25fa68d65485d63d79eda"}
Jul 07 19:09:32.665490 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.665472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2" event={"ID":"9832adb6-fada-441b-b0ce-afadc0758567","Type":"ContainerStarted","Data":"a313d12e7731598eae785c3ab0d0e4241777f350ec09fd0e0002df17941f68dd"}
Jul 07 19:09:32.666929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.666905    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3cfd0e1ebd8169b57d9e712a094a397" containerID="e0a5eadb9d9c97fdda6e83835249e4fb14cc9d752f5eeb2832ee04fd1aae3ba8" exitCode=0
Jul 07 19:09:32.667034 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.666989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal" event={"ID":"d3cfd0e1ebd8169b57d9e712a094a397","Type":"ContainerDied","Data":"e0a5eadb9d9c97fdda6e83835249e4fb14cc9d752f5eeb2832ee04fd1aae3ba8"}
Jul 07 19:09:32.673912 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.673877    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v9nhr" podStartSLOduration=12.575986476 podStartE2EDuration="23.673864814s" totalImagesPullingTime="11.097878338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:32.673503716 +0000 UTC m=+23.634954993" watchObservedRunningTime="2026-07-07 19:09:32.673864814 +0000 UTC m=+23.635316126"
Jul 07 19:09:32.691305 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.691273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-djljf" podStartSLOduration=4.009544336 podStartE2EDuration="23.691261441s" totalImagesPullingTime="19.681717105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:32.691116097 +0000 UTC m=+23.652567374" watchObservedRunningTime="2026-07-07 19:09:32.691261441 +0000 UTC m=+23.652712719"
Jul 07 19:09:32.709870 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:32.709823    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-85qcf" podStartSLOduration=4.039263418 podStartE2EDuration="23.709807767s" totalImagesPullingTime="19.670544349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:32.709500711 +0000 UTC m=+23.670951988" watchObservedRunningTime="2026-07-07 19:09:32.709807767 +0000 UTC m=+23.671259045"
Jul 07 19:09:33.542055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.542018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:33.542237 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:33.542152    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:33.542572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.542550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:33.542958 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:33.542665    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:33.671125 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.671089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal" event={"ID":"d3cfd0e1ebd8169b57d9e712a094a397","Type":"ContainerStarted","Data":"2d507df29626b8882154889939068fff1552a236dfed661e70929d6c2c022e35"}
Jul 07 19:09:33.687326 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.687252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-217.ec2.internal" podStartSLOduration=23.687232344999998 podStartE2EDuration="23.687232345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:33.687029819 +0000 UTC m=+24.648481096" watchObservedRunningTime="2026-07-07 19:09:33.687232345 +0000 UTC m=+24.648683622"
Jul 07 19:09:33.687732 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.687703    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l69s9" podStartSLOduration=5.037154777 podStartE2EDuration="24.687691217s" totalImagesPullingTime="19.65053644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:32.767442212 +0000 UTC m=+23.728893489" watchObservedRunningTime="2026-07-07 19:09:33.687691217 +0000 UTC m=+24.649142494"
Jul 07 19:09:33.854291 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:33.854269    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 19:09:34.415081 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.414970    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T19:09:33.854284941Z","UUID":"c1015e70-ee51-4369-859e-90c43cfe8d5e","Handler":null,"Name":"","Endpoint":""}
Jul 07 19:09:34.416863 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.416835    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 19:09:34.416999 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.416871    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 19:09:34.542800 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.542768    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:34.543271 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:34.542893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:34.675205 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.675160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2" event={"ID":"9832adb6-fada-441b-b0ce-afadc0758567","Type":"ContainerStarted","Data":"c81247383fe1423bb4d1a006ce5e56c98253af2cd9e9f474e4b85ecfe74305fa"}
Jul 07 19:09:34.678536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:34.678512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"ca960c07f3d20954d2c1d2cf81088a333317e8549dd84a79239c8c8c69a1a0b0"}
Jul 07 19:09:35.542611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:35.542563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:35.542796 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:35.542685    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:35.542796 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:35.542740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:35.543086 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:35.542830    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:35.682744 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:35.682705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2" event={"ID":"9832adb6-fada-441b-b0ce-afadc0758567","Type":"ContainerStarted","Data":"226f61bec05c0e9b5d932a51f1cebd3d7d8d9c3cf56ed4caeedb53e83f56178a"}
Jul 07 19:09:36.123284 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.123249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:36.123780 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.123761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:36.141043 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.141004    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sqqx2" podStartSLOduration=4.871417701 podStartE2EDuration="27.140986002s" totalImagesPullingTime="22.269568301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:35.70367764 +0000 UTC m=+26.665128916" watchObservedRunningTime="2026-07-07 19:09:36.140986002 +0000 UTC m=+27.102437280"
Jul 07 19:09:36.542046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.541907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:36.542165 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:36.542146    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:36.689155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.689067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" event={"ID":"b6cba104-1d0b-4caa-ae20-ee37d44d6fdb","Type":"ContainerStarted","Data":"2ff6d88f8c29ae706d8c86b2f26ab6226f3be83aa6fa0880d13d5ad45bd2ea54"}
Jul 07 19:09:36.689738 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.689605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:36.689835 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.689817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v9nhr"
Jul 07 19:09:36.718474 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:36.718430    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-82trz" podStartSLOduration=7.9512764350000005 podStartE2EDuration="27.71841719s" totalImagesPullingTime="19.767140755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:36.718204826 +0000 UTC m=+27.679656095" watchObservedRunningTime="2026-07-07 19:09:36.71841719 +0000 UTC m=+27.679868486"
Jul 07 19:09:37.542466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.542435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:37.542665 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:37.542539    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:37.542665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.542610    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:37.542819 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:37.542708    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:37.700556 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.700524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:37.701088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.700630    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:37.701088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.700675    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:37.718555 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.718529    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:37.719234 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:37.719214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:09:38.542406 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.542377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:38.542565 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:38.542510    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:38.745873 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.745841    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wp8nm"]
Jul 07 19:09:38.746513 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.745941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:38.746513 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:38.746049    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:38.750400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.750373    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xrb6"]
Jul 07 19:09:38.750515 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.750468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:38.750613 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:38.750576    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:38.762659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.762624    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n2tls"]
Jul 07 19:09:38.762798 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:38.762716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:38.762865 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:38.762820    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:40.542367 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:40.542175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:40.542722 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:40.542211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:40.542722 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:40.542439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:40.542722 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:40.542246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:40.542722 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:40.542502    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:40.542722 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:40.542617    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:40.709243 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:40.709171    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="0cb651595f5460256a8048b16e80c906c2304d7a82ea2fefa1a8a29309bbec14" exitCode=0
Jul 07 19:09:40.709243 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:40.709218    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"0cb651595f5460256a8048b16e80c906c2304d7a82ea2fefa1a8a29309bbec14"}
Jul 07 19:09:42.542511 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:42.542474    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:42.542511 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:42.542514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:42.542919 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:42.542612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:42.542919 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:42.542614    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:42.542919 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:42.542687    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:42.542919 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:42.542773    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:42.716308 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:42.716274    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="e0f5e186d149882defaeceb8846b7c81a7ddc89de7b3599624d90d1e8a03b0ec" exitCode=0
Jul 07 19:09:42.716481 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:42.716342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"e0f5e186d149882defaeceb8846b7c81a7ddc89de7b3599624d90d1e8a03b0ec"}
Jul 07 19:09:43.110385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:43.110307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:43.110511 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.110441    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:43.110511 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.110497    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:15.110482324 +0000 UTC m=+66.071933580 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 19:09:43.211122 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:43.211094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:43.211248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:43.211150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:43.211248 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211226    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:43.211248 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211237    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 19:09:43.211335 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211256    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 19:09:43.211335 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211265    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mr8hj for pod openshift-network-diagnostics/network-check-target-n2tls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:43.211335 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211271    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret podName:fb0b3d25-d5e0-42bd-9eff-9909e26175d5 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:15.211258481 +0000 UTC m=+66.172709736 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret") pod "global-pull-secret-syncer-wp8nm" (UID: "fb0b3d25-d5e0-42bd-9eff-9909e26175d5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 19:09:43.211335 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:43.211300    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj podName:e037cf30-db78-4228-8caa-b80762fb15cf nodeName:}" failed. No retries permitted until 2026-07-07 19:10:15.21128869 +0000 UTC m=+66.172739945 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mr8hj" (UniqueName: "kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj") pod "network-check-target-n2tls" (UID: "e037cf30-db78-4228-8caa-b80762fb15cf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 19:09:44.542673 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:44.542637    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:44.542673 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:44.542669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:44.543090 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:44.542717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:44.543090 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:44.542725    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n2tls" podUID="e037cf30-db78-4228-8caa-b80762fb15cf"
Jul 07 19:09:44.543090 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:44.542790    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xrb6" podUID="dc8237c9-6a76-49c4-b86c-818f95b3aa82"
Jul 07 19:09:44.543090 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:44.542864    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wp8nm" podUID="fb0b3d25-d5e0-42bd-9eff-9909e26175d5"
Jul 07 19:09:44.722960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:44.722927    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="84af002a9676ba8cd3d4e3333f360957d63d3988092517567771d9b70d1072c5" exitCode=0
Jul 07 19:09:44.723091 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:44.722970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"84af002a9676ba8cd3d4e3333f360957d63d3988092517567771d9b70d1072c5"}
Jul 07 19:09:45.324812 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.324638    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-217.ec2.internal" event="NodeReady"
Jul 07 19:09:45.325043 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.324927    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 19:09:45.382063 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.382030    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:09:45.384928 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.384912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.386975 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.386947    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 19:09:45.386975 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.386967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 19:09:45.387122 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.387038    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 19:09:45.387122 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.387072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xhv4z\""
Jul 07 19:09:45.393016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.392995    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 19:09:45.407698 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.407677    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:09:45.410210 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.410190    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-whk7g"]
Jul 07 19:09:45.413516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.413496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.415707 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.415692    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g92mm\""
Jul 07 19:09:45.415812 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.415710    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 19:09:45.416206 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.416189    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 19:09:45.434557 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.434536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-whk7g"]
Jul 07 19:09:45.443889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.443868    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g58w5"]
Jul 07 19:09:45.446745 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.446726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.450680 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.450660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 19:09:45.452350 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.452087    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 19:09:45.452350 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.452157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nhxfv\""
Jul 07 19:09:45.452350 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.452166    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 19:09:45.468563 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.468536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g58w5"]
Jul 07 19:09:45.528416 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528567 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528567 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528567 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae84ed10-ccc5-4a89-9374-bbbcda75246a-config-volume\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcplt\" (UniqueName: \"kubernetes.io/projected/ae84ed10-ccc5-4a89-9374-bbbcda75246a-kube-api-access-kcplt\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzpmm\" (UniqueName: \"kubernetes.io/projected/720a723a-49e6-4c43-9b03-dfef193e6930-kube-api-access-zzpmm\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/720a723a-49e6-4c43-9b03-dfef193e6930-cert\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24bpx\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae84ed10-ccc5-4a89-9374-bbbcda75246a-tmp-dir\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.528755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.529151 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.528774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.629238 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629238 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629238 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/720a723a-49e6-4c43-9b03-dfef193e6930-cert\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629285    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24bpx\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae84ed10-ccc5-4a89-9374-bbbcda75246a-tmp-dir\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:45.629328    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:45.629343    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:45.629412    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:46.129392527 +0000 UTC m=+37.090843784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:45.629753    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:09:45.629769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae84ed10-ccc5-4a89-9374-bbbcda75246a-config-volume\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.630347 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcplt\" (UniqueName: \"kubernetes.io/projected/ae84ed10-ccc5-4a89-9374-bbbcda75246a-kube-api-access-kcplt\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.630347 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:45.629816    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:09:46.129798858 +0000 UTC m=+37.091250133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:09:45.630347 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzpmm\" (UniqueName: \"kubernetes.io/projected/720a723a-49e6-4c43-9b03-dfef193e6930-kube-api-access-zzpmm\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.630347 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629893    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.630347 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.629893    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae84ed10-ccc5-4a89-9374-bbbcda75246a-tmp-dir\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.630561 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.630419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.630561 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.630461    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae84ed10-ccc5-4a89-9374-bbbcda75246a-config-volume\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.630561 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.630471    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.633580 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.633555    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.633702 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.633636    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.633702 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.633651    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/720a723a-49e6-4c43-9b03-dfef193e6930-cert\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.646246 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.646221    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24bpx\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.647249 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.647221    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:45.647647 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.647625    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcplt\" (UniqueName: \"kubernetes.io/projected/ae84ed10-ccc5-4a89-9374-bbbcda75246a-kube-api-access-kcplt\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:45.647721 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.647707    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzpmm\" (UniqueName: \"kubernetes.io/projected/720a723a-49e6-4c43-9b03-dfef193e6930-kube-api-access-zzpmm\") pod \"ingress-canary-g58w5\" (UID: \"720a723a-49e6-4c43-9b03-dfef193e6930\") " pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.756779 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.756751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g58w5"
Jul 07 19:09:45.904992 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:45.904927    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g58w5"]
Jul 07 19:09:45.908322 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:09:45.908295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720a723a_49e6_4c43_9b03_dfef193e6930.slice/crio-2f8e86e2bf75d2a02f6e2d24c0d50cbc79bd0f8a80084e43b86ad5e7467173a9 WatchSource:0}: Error finding container 2f8e86e2bf75d2a02f6e2d24c0d50cbc79bd0f8a80084e43b86ad5e7467173a9: Status 404 returned error can't find the container with id 2f8e86e2bf75d2a02f6e2d24c0d50cbc79bd0f8a80084e43b86ad5e7467173a9
Jul 07 19:09:46.133010 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.132979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:46.133172 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.133026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:46.133172 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:46.133147    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:09:46.133172 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:46.133146    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:09:46.133172 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:46.133169    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:09:46.133327 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:46.133200    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:09:47.133186334 +0000 UTC m=+38.094637588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:09:46.133327 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:46.133224    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:47.133209584 +0000 UTC m=+38.094660839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:09:46.542787 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.542732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:09:46.542787 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.542769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:09:46.543045 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.542732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:09:46.545039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545017    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 19:09:46.545138 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 19:09:46.545138 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 19:09:46.545445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545427    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jqhn7\""
Jul 07 19:09:46.545540 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545514    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 19:09:46.545540 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.545537    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkm4d\""
Jul 07 19:09:46.730717 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:46.730680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g58w5" event={"ID":"720a723a-49e6-4c43-9b03-dfef193e6930","Type":"ContainerStarted","Data":"2f8e86e2bf75d2a02f6e2d24c0d50cbc79bd0f8a80084e43b86ad5e7467173a9"}
Jul 07 19:09:47.139997 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:47.139956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:47.140174 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:47.140085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:47.140174 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:47.140152    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:09:47.140267 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:47.140222    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:09:47.140267 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:47.140236    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:09:47.140340 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:47.140236    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:09:49.140211928 +0000 UTC m=+40.101663186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:09:47.140340 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:47.140307    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:49.140289154 +0000 UTC m=+40.101740417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:09:49.158173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:49.158117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:49.158736 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:49.158187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:49.158736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:49.158377    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:09:49.158736 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:49.158435    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:09:53.158421723 +0000 UTC m=+44.119872982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:09:49.159086 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:49.159062    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:09:49.159158 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:49.159099    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:09:49.159207 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:49.159172    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:09:53.159153071 +0000 UTC m=+44.120604340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:09:52.747767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:52.747563    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="68c967c381ed1e8f30fa104cf35625fabc9b609f2da42ea88ee13df1de3c93b4" exitCode=0
Jul 07 19:09:52.748186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:52.747653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"68c967c381ed1e8f30fa104cf35625fabc9b609f2da42ea88ee13df1de3c93b4"}
Jul 07 19:09:52.749359 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:52.749334    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g58w5" event={"ID":"720a723a-49e6-4c43-9b03-dfef193e6930","Type":"ContainerStarted","Data":"7a2b0e8b839c1456b4e5a69c3d91c977788e024171ae5a4146876ce0195e0157"}
Jul 07 19:09:52.785773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:52.785218    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g58w5" podStartSLOduration=1.520519072 podStartE2EDuration="7.785200697s" totalImagesPullingTime="6.264681625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:52.785084248 +0000 UTC m=+43.746535527" watchObservedRunningTime="2026-07-07 19:09:52.785200697 +0000 UTC m=+43.746651974"
Jul 07 19:09:53.193005 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:53.192922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:09:53.193005 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:53.192966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:09:53.193194 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:53.193083    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:09:53.193194 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:53.193112    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:09:53.193194 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:53.193166    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:01.193151034 +0000 UTC m=+52.154602289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:09:53.193194 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:53.193089    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:09:53.193344 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:09:53.193218    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:10:01.193203423 +0000 UTC m=+52.154654678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:09:53.754358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:53.754324    2579 generic.go:356] "Generic (PLEG): container finished" podID="3781fe31-12a2-450f-9331-ff8740b14849" containerID="dec08b60c9c373c566c700ce76afa4c3031a1893210e1025e79defd3238f50f9" exitCode=0
Jul 07 19:09:53.754753 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:53.754404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerDied","Data":"dec08b60c9c373c566c700ce76afa4c3031a1893210e1025e79defd3238f50f9"}
Jul 07 19:09:54.759529 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:54.759488    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb76f" event={"ID":"3781fe31-12a2-450f-9331-ff8740b14849","Type":"ContainerStarted","Data":"fd20fcf002f702aa042162e6fa6db3f0b6eb4ccc0664ca6de35211fa29342ab2"}
Jul 07 19:09:54.789055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:09:54.789001    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hb76f" podStartSLOduration=9.452597222 podStartE2EDuration="45.788981241s" totalImagesPullingTime="36.336384019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:54.788443589 +0000 UTC m=+45.749894865" watchObservedRunningTime="2026-07-07 19:09:54.788981241 +0000 UTC m=+45.750432519"
Jul 07 19:10:01.249279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:01.249236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:01.249288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:01.249379    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:01.249385    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:01.249401    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:01.249439    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:10:17.249426569 +0000 UTC m=+68.210877823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:10:01.249752 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:01.249453    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:17.249446338 +0000 UTC m=+68.210897592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:10:09.118325 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.118294    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"]
Jul 07 19:10:09.123295 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.123280    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.127081 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.127032    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 19:10:09.127210 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.127190    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 19:10:09.127279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.127211    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69xk7\""
Jul 07 19:10:09.127478 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.127461    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 19:10:09.128212 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.128198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 19:10:09.139758 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.139738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"]
Jul 07 19:10:09.202763 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.202738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.202889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.202765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppt69\" (UniqueName: \"kubernetes.io/projected/c5bceb52-bdd2-413e-8871-dcefa851d0d9-kube-api-access-ppt69\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.202889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.202797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5bceb52-bdd2-413e-8871-dcefa851d0d9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.220441 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.220416    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"]
Jul 07 19:10:09.223399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.223385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.225754 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.225736    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 19:10:09.226262 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.226245    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8vmkz\""
Jul 07 19:10:09.226581 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.226566    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 19:10:09.226904 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.226879    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 19:10:09.238601 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.238564    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"]
Jul 07 19:10:09.303927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.303896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.304067 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.303942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zdb\" (UniqueName: \"kubernetes.io/projected/33859ecf-f267-4bda-978f-3a0e3aa4c617-kube-api-access-z6zdb\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.304067 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.304026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.304067 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.304053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppt69\" (UniqueName: \"kubernetes.io/projected/c5bceb52-bdd2-413e-8871-dcefa851d0d9-kube-api-access-ppt69\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.304165 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.304084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5bceb52-bdd2-413e-8871-dcefa851d0d9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.304165 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.304117    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:09.304236 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.304190    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:09.804171855 +0000 UTC m=+60.765623115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:09.304748 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.304732    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5bceb52-bdd2-413e-8871-dcefa851d0d9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.317172 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.317146    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppt69\" (UniqueName: \"kubernetes.io/projected/c5bceb52-bdd2-413e-8871-dcefa851d0d9-kube-api-access-ppt69\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.404363 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.404299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.404363 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.404340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zdb\" (UniqueName: \"kubernetes.io/projected/33859ecf-f267-4bda-978f-3a0e3aa4c617-kube-api-access-z6zdb\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.406530 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.406511    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 19:10:09.413308 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.413281    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 19:10:09.414491 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.414473    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 19:10:09.414608 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.414535    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls podName:33859ecf-f267-4bda-978f-3a0e3aa4c617 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:09.914517695 +0000 UTC m=+60.875968950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qbnbd" (UID: "33859ecf-f267-4bda-978f-3a0e3aa4c617") : secret "samples-operator-tls" not found
Jul 07 19:10:09.423787 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.423770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 19:10:09.434856 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.434829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zdb\" (UniqueName: \"kubernetes.io/projected/33859ecf-f267-4bda-978f-3a0e3aa4c617-kube-api-access-z6zdb\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:09.723289 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.723264    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82trz"
Jul 07 19:10:09.806465 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:09.806431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:09.806647 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.806508    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:09.806647 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:09.806555    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:10.806538812 +0000 UTC m=+61.767990067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:10.008350 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:10.008267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:10.008496 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:10.008380    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 19:10:10.008496 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:10.008442    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls podName:33859ecf-f267-4bda-978f-3a0e3aa4c617 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:11.008428316 +0000 UTC m=+61.969879571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qbnbd" (UID: "33859ecf-f267-4bda-978f-3a0e3aa4c617") : secret "samples-operator-tls" not found
Jul 07 19:10:10.812906 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:10.812875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:10.813291 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:10.813011    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:10.813291 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:10.813069    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:12.813054796 +0000 UTC m=+63.774506050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:11.014312 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:11.014273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:11.014482 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:11.014388    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 19:10:11.014482 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:11.014449    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls podName:33859ecf-f267-4bda-978f-3a0e3aa4c617 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:13.014431325 +0000 UTC m=+63.975882593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qbnbd" (UID: "33859ecf-f267-4bda-978f-3a0e3aa4c617") : secret "samples-operator-tls" not found
Jul 07 19:10:12.827163 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:12.827126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:12.827547 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:12.827263    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:12.827547 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:12.827322    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:16.82730681 +0000 UTC m=+67.788758065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:13.028900 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.028865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:13.029049 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.029028    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 19:10:13.029118 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.029109    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls podName:33859ecf-f267-4bda-978f-3a0e3aa4c617 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:17.029093223 +0000 UTC m=+67.990544478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qbnbd" (UID: "33859ecf-f267-4bda-978f-3a0e3aa4c617") : secret "samples-operator-tls" not found
Jul 07 19:10:13.149203 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.149133    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c5jz4"]
Jul 07 19:10:13.151572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.151557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.153794 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.153764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f8xvk\""
Jul 07 19:10:13.153933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.153888    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 19:10:13.153933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.153902    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 19:10:13.153933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.153912    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 19:10:13.153933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.153907    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 19:10:13.167952 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.167929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5jz4"]
Jul 07 19:10:13.230641 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.230606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfhh\" (UniqueName: \"kubernetes.io/projected/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-api-access-ztfhh\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.230791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.230660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4156ab8-26bf-4357-bab3-289a19fd4bb4-crio-socket\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.230791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.230762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4156ab8-26bf-4357-bab3-289a19fd4bb4-data-volume\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.230791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.230786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.230968 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.230807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331354 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4156ab8-26bf-4357-bab3-289a19fd4bb4-data-volume\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331354 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.331454    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331466    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfhh\" (UniqueName: \"kubernetes.io/projected/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-api-access-ztfhh\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4156ab8-26bf-4357-bab3-289a19fd4bb4-crio-socket\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.331530    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:13.831516099 +0000 UTC m=+64.792967354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : configmap "kube-rbac-proxy" not found
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.331552    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:13.331615 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331575    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4156ab8-26bf-4357-bab3-289a19fd4bb4-crio-socket\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.331873 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.331647    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:13.831630847 +0000 UTC m=+64.793082108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:13.331873 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.331731    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4156ab8-26bf-4357-bab3-289a19fd4bb4-data-volume\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.346425 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.346397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfhh\" (UniqueName: \"kubernetes.io/projected/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-api-access-ztfhh\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.834729 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.834698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.834729 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:13.834734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:13.835116 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.834821    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 19:10:13.835116 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.834880    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:14.834863012 +0000 UTC m=+65.796314269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : configmap "kube-rbac-proxy" not found
Jul 07 19:10:13.835116 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.834920    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:13.835116 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:13.835011    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:14.834997889 +0000 UTC m=+65.796449144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:14.842756 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:14.842719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:14.842756 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:14.842758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:14.843264 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:14.842835    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 19:10:14.843264 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:14.842911    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:16.842893568 +0000 UTC m=+67.804344828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : configmap "kube-rbac-proxy" not found
Jul 07 19:10:14.843264 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:14.842955    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:14.843264 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:14.843017    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:16.843000959 +0000 UTC m=+67.804452217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:15.145740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.145646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:10:15.147763 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.147743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 19:10:15.156177 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:15.156157    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 19:10:15.156281 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:15.156226    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs podName:dc8237c9-6a76-49c4-b86c-818f95b3aa82 nodeName:}" failed. No retries permitted until 2026-07-07 19:11:19.156203911 +0000 UTC m=+130.117655169 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs") pod "network-metrics-daemon-5xrb6" (UID: "dc8237c9-6a76-49c4-b86c-818f95b3aa82") : secret "metrics-daemon-secret" not found
Jul 07 19:10:15.247109 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.247074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:10:15.247267 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.247131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:10:15.249128 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.249105    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 19:10:15.249193 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.249171    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 19:10:15.260012 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.259993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 19:10:15.260270 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.260249    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb0b3d25-d5e0-42bd-9eff-9909e26175d5-original-pull-secret\") pod \"global-pull-secret-syncer-wp8nm\" (UID: \"fb0b3d25-d5e0-42bd-9eff-9909e26175d5\") " pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:10:15.287767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.287734    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8hj\" (UniqueName: \"kubernetes.io/projected/e037cf30-db78-4228-8caa-b80762fb15cf-kube-api-access-mr8hj\") pod \"network-check-target-n2tls\" (UID: \"e037cf30-db78-4228-8caa-b80762fb15cf\") " pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:10:15.357672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.357639    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l69s9_5efe5858-ff97-4429-b89f-85e7cc3cf1a4/dns-node-resolver/0.log"
Jul 07 19:10:15.363721 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.363699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkm4d\""
Jul 07 19:10:15.365197 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.365182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wp8nm"
Jul 07 19:10:15.371968 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.371946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:10:15.495048 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.495021    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wp8nm"]
Jul 07 19:10:15.500701 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:15.499693    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0b3d25_d5e0_42bd_9eff_9909e26175d5.slice/crio-c3f97f1575dbc1c0a5fb06c0eec895d8df51d3a3d5d69467a29833da1d27d727 WatchSource:0}: Error finding container c3f97f1575dbc1c0a5fb06c0eec895d8df51d3a3d5d69467a29833da1d27d727: Status 404 returned error can't find the container with id c3f97f1575dbc1c0a5fb06c0eec895d8df51d3a3d5d69467a29833da1d27d727
Jul 07 19:10:15.509126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.509102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n2tls"]
Jul 07 19:10:15.511838 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:15.511812    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode037cf30_db78_4228_8caa_b80762fb15cf.slice/crio-fbe84c8c0024da2abb6960b0599c3a65b1b42c166b4b5a653d7ed9ef8399a31f WatchSource:0}: Error finding container fbe84c8c0024da2abb6960b0599c3a65b1b42c166b4b5a653d7ed9ef8399a31f: Status 404 returned error can't find the container with id fbe84c8c0024da2abb6960b0599c3a65b1b42c166b4b5a653d7ed9ef8399a31f
Jul 07 19:10:15.809983 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.809950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n2tls" event={"ID":"e037cf30-db78-4228-8caa-b80762fb15cf","Type":"ContainerStarted","Data":"fbe84c8c0024da2abb6960b0599c3a65b1b42c166b4b5a653d7ed9ef8399a31f"}
Jul 07 19:10:15.810986 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.810962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wp8nm" event={"ID":"fb0b3d25-d5e0-42bd-9eff-9909e26175d5","Type":"ContainerStarted","Data":"c3f97f1575dbc1c0a5fb06c0eec895d8df51d3a3d5d69467a29833da1d27d727"}
Jul 07 19:10:15.959948 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:15.959917    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-85qcf_ed443871-0f3a-487a-afe6-38fe7fc9ae81/node-ca/0.log"
Jul 07 19:10:16.760933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:16.760871    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g58w5_720a723a-49e6-4c43-9b03-dfef193e6930/serve-healthcheck-canary/0.log"
Jul 07 19:10:16.860575 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:16.860535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:16.860781 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:16.860634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:16.860781 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:16.860680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:16.860781 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.860723    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:16.861290 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.861259    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:24.861111786 +0000 UTC m=+75.822563056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:16.864987 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.864957    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:16.865099 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.864994    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 19:10:16.865099 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.865047    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:20.865027555 +0000 UTC m=+71.826478815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:16.865099 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:16.865070    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:20.865059554 +0000 UTC m=+71.826510809 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : configmap "kube-rbac-proxy" not found
Jul 07 19:10:17.062331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:17.062235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:17.062809 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.062433    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 19:10:17.062809 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.062541    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls podName:33859ecf-f267-4bda-978f-3a0e3aa4c617 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:25.062519727 +0000 UTC m=+76.023970995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qbnbd" (UID: "33859ecf-f267-4bda-978f-3a0e3aa4c617") : secret "samples-operator-tls" not found
Jul 07 19:10:17.264984 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:17.264943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:17.265171 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:17.265010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:17.265171 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.265118    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 19:10:17.265171 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.265163    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b6656699-7jr5k: secret "image-registry-tls" not found
Jul 07 19:10:17.265353 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.265173    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 19:10:17.265353 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.265235    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls podName:06b0f316-b64a-4015-8662-6fa3d051cff8 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:49.265214403 +0000 UTC m=+100.226665678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls") pod "image-registry-9b6656699-7jr5k" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8") : secret "image-registry-tls" not found
Jul 07 19:10:17.265353 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:17.265255    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls podName:ae84ed10-ccc5-4a89-9374-bbbcda75246a nodeName:}" failed. No retries permitted until 2026-07-07 19:10:49.265246354 +0000 UTC m=+100.226697623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls") pod "dns-default-whk7g" (UID: "ae84ed10-ccc5-4a89-9374-bbbcda75246a") : secret "dns-default-metrics-tls" not found
Jul 07 19:10:19.824608 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:19.824556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n2tls" event={"ID":"e037cf30-db78-4228-8caa-b80762fb15cf","Type":"ContainerStarted","Data":"39493f9b53209b057f9c64e64ab489a17bf4726ee3bdcf170688a11354cb7473"}
Jul 07 19:10:19.825066 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:19.824660    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:10:19.840952 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:19.840902    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n2tls" podStartSLOduration=67.403135541 podStartE2EDuration="1m10.840884173s" totalImagesPullingTime="3.437748632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:19.840729469 +0000 UTC m=+70.802180748" watchObservedRunningTime="2026-07-07 19:10:19.840884173 +0000 UTC m=+70.802335444"
Jul 07 19:10:20.828437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:20.828400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wp8nm" event={"ID":"fb0b3d25-d5e0-42bd-9eff-9909e26175d5","Type":"ContainerStarted","Data":"70bb35bb97af31993524061d3036d50cfacee150865fbf7f95bdeda26f37c553"}
Jul 07 19:10:20.897207 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:20.897178    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:20.897207 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:20.897206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:20.897408 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:20.897316    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 19:10:20.897408 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:20.897339    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:20.897408 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:20.897383    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:28.897365967 +0000 UTC m=+79.858817237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : configmap "kube-rbac-proxy" not found
Jul 07 19:10:20.897408 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:20.897398    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls podName:a4156ab8-26bf-4357-bab3-289a19fd4bb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:28.897391379 +0000 UTC m=+79.858842634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5jz4" (UID: "a4156ab8-26bf-4357-bab3-289a19fd4bb4") : secret "insights-runtime-extractor-tls" not found
Jul 07 19:10:24.928228 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:24.928186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:24.928687 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:24.928340    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:24.928687 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:24.928407    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls podName:c5bceb52-bdd2-413e-8871-dcefa851d0d9 nodeName:}" failed. No retries permitted until 2026-07-07 19:10:40.928391293 +0000 UTC m=+91.889842548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hw4cb" (UID: "c5bceb52-bdd2-413e-8871-dcefa851d0d9") : secret "cluster-monitoring-operator-tls" not found
Jul 07 19:10:25.129731 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.129695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:25.132258 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.132222    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33859ecf-f267-4bda-978f-3a0e3aa4c617-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qbnbd\" (UID: \"33859ecf-f267-4bda-978f-3a0e3aa4c617\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:25.433598 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.433567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8vmkz\""
Jul 07 19:10:25.441959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.441940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"
Jul 07 19:10:25.560448 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.560342    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wp8nm" podStartSLOduration=71.652567301 podStartE2EDuration="1m16.56032143s" totalImagesPullingTime="4.907754129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:20.843184053 +0000 UTC m=+71.804635329" watchObservedRunningTime="2026-07-07 19:10:25.56032143 +0000 UTC m=+76.521772701"
Jul 07 19:10:25.561378 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.561354    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd"]
Jul 07 19:10:25.841745 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:25.841709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd" event={"ID":"33859ecf-f267-4bda-978f-3a0e3aa4c617","Type":"ContainerStarted","Data":"fe9aad6b13ad5a64ee0201bdcaf903701be1000d3b354f601618f8a1e53d5f52"}
Jul 07 19:10:28.851233 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.851193    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd" event={"ID":"33859ecf-f267-4bda-978f-3a0e3aa4c617","Type":"ContainerStarted","Data":"89ad7ec6ed0fce4fa4d4dcde20b3e6c50f2172ad1ecebdf85574c5ea014c038b"}
Jul 07 19:10:28.851662 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.851293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd" event={"ID":"33859ecf-f267-4bda-978f-3a0e3aa4c617","Type":"ContainerStarted","Data":"52b21c11cb524370936c5f6cc5d3f5b80ffc88ebc89c66e7f85685e25c57952c"}
Jul 07 19:10:28.960445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.960410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:28.960445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.960443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:28.961113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.961082    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4156ab8-26bf-4357-bab3-289a19fd4bb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:28.962955 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:28.962927    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4156ab8-26bf-4357-bab3-289a19fd4bb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5jz4\" (UID: \"a4156ab8-26bf-4357-bab3-289a19fd4bb4\") " pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:29.060597 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:29.060564    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5jz4"
Jul 07 19:10:29.184858 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:29.184807    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qbnbd" podStartSLOduration=17.863029274 podStartE2EDuration="20.184787009s" totalImagesPullingTime="2.321757735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:28.871739351 +0000 UTC m=+79.833190629" watchObservedRunningTime="2026-07-07 19:10:29.184787009 +0000 UTC m=+80.146238285"
Jul 07 19:10:29.185210 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:29.185196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5jz4"]
Jul 07 19:10:29.187894 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:29.187869    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4156ab8_26bf_4357_bab3_289a19fd4bb4.slice/crio-b081c3524859dfc6475c9e85bcadef153b1c02a59f0324afe58eb1a8bb999cad WatchSource:0}: Error finding container b081c3524859dfc6475c9e85bcadef153b1c02a59f0324afe58eb1a8bb999cad: Status 404 returned error can't find the container with id b081c3524859dfc6475c9e85bcadef153b1c02a59f0324afe58eb1a8bb999cad
Jul 07 19:10:29.857878 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:29.857653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5jz4" event={"ID":"a4156ab8-26bf-4357-bab3-289a19fd4bb4","Type":"ContainerStarted","Data":"b081c3524859dfc6475c9e85bcadef153b1c02a59f0324afe58eb1a8bb999cad"}
Jul 07 19:10:32.870368 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:32.870342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5jz4" event={"ID":"a4156ab8-26bf-4357-bab3-289a19fd4bb4","Type":"ContainerStarted","Data":"dc5cec15d524a334773efc908c33c7a2e178160f1320811d632b7ba015415ac7"}
Jul 07 19:10:33.874772 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:33.874692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5jz4" event={"ID":"a4156ab8-26bf-4357-bab3-289a19fd4bb4","Type":"ContainerStarted","Data":"8133d76a0ad1d0c96ecc38880728592518b45ab590e332c9d264bc9c39062f4e"}
Jul 07 19:10:36.886331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:36.886294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5jz4" event={"ID":"a4156ab8-26bf-4357-bab3-289a19fd4bb4","Type":"ContainerStarted","Data":"c0907c2599a5e532e0ba8ea76491df19a473dae5bce5b1689cb8bd2a37d6b26c"}
Jul 07 19:10:36.906770 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:36.906723    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c5jz4" podStartSLOduration=16.965197613 podStartE2EDuration="23.906707565s" totalImagesPullingTime="6.941509952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:10:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:36.905579604 +0000 UTC m=+87.867030882" watchObservedRunningTime="2026-07-07 19:10:36.906707565 +0000 UTC m=+87.868158841"
Jul 07 19:10:40.955164 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:40.955130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:40.957564 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:40.957547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5bceb52-bdd2-413e-8871-dcefa851d0d9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hw4cb\" (UID: \"c5bceb52-bdd2-413e-8871-dcefa851d0d9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:41.234034 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.234009    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69xk7\""
Jul 07 19:10:41.242528 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.242509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"
Jul 07 19:10:41.365158 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.365139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb"]
Jul 07 19:10:41.367556 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:41.367525    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bceb52_bdd2_413e_8871_dcefa851d0d9.slice/crio-8e296c3c4258f2ec4c0e2eb0ff89d00b806621d14067f463b2caaa0b565aefb5 WatchSource:0}: Error finding container 8e296c3c4258f2ec4c0e2eb0ff89d00b806621d14067f463b2caaa0b565aefb5: Status 404 returned error can't find the container with id 8e296c3c4258f2ec4c0e2eb0ff89d00b806621d14067f463b2caaa0b565aefb5
Jul 07 19:10:41.592779 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.592709    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"]
Jul 07 19:10:41.596503 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.596486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.598737 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.598714    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 19:10:41.598840 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.598791    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 19:10:41.599061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.599043    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 19:10:41.599143 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.599125    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 19:10:41.611550 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.611529    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"]
Jul 07 19:10:41.661809 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.661779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8f8\" (UniqueName: \"kubernetes.io/projected/8a51f612-c192-4159-88ba-00234127b85d-kube-api-access-dj8f8\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.661931 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.661826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a51f612-c192-4159-88ba-00234127b85d-klusterlet-config\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.661931 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.661922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a51f612-c192-4159-88ba-00234127b85d-tmp\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.691909 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.691887    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-xc9kj"]
Jul 07 19:10:41.694702 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.694687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:10:41.696558 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.696541    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-flq7k\""
Jul 07 19:10:41.696797 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.696779    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 19:10:41.696896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.696879    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 19:10:41.705255 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.705237    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xc9kj"]
Jul 07 19:10:41.763279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.763257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8f8\" (UniqueName: \"kubernetes.io/projected/8a51f612-c192-4159-88ba-00234127b85d-kube-api-access-dj8f8\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.763385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.763289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a51f612-c192-4159-88ba-00234127b85d-klusterlet-config\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.763385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.763309    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwdlr\" (UniqueName: \"kubernetes.io/projected/11398cfd-4692-40d0-bf1b-f84414c94184-kube-api-access-bwdlr\") pod \"downloads-685bfb5f64-xc9kj\" (UID: \"11398cfd-4692-40d0-bf1b-f84414c94184\") " pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:10:41.763385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.763351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a51f612-c192-4159-88ba-00234127b85d-tmp\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.763739 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.763719    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a51f612-c192-4159-88ba-00234127b85d-tmp\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.766092 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.766069    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a51f612-c192-4159-88ba-00234127b85d-klusterlet-config\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.777006 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.776981    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8f8\" (UniqueName: \"kubernetes.io/projected/8a51f612-c192-4159-88ba-00234127b85d-kube-api-access-dj8f8\") pod \"klusterlet-addon-workmgr-f985dc686-ldfg6\" (UID: \"8a51f612-c192-4159-88ba-00234127b85d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:41.864116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.864038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwdlr\" (UniqueName: \"kubernetes.io/projected/11398cfd-4692-40d0-bf1b-f84414c94184-kube-api-access-bwdlr\") pod \"downloads-685bfb5f64-xc9kj\" (UID: \"11398cfd-4692-40d0-bf1b-f84414c94184\") " pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:10:41.873652 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.873626    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwdlr\" (UniqueName: \"kubernetes.io/projected/11398cfd-4692-40d0-bf1b-f84414c94184-kube-api-access-bwdlr\") pod \"downloads-685bfb5f64-xc9kj\" (UID: \"11398cfd-4692-40d0-bf1b-f84414c94184\") " pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:10:41.902019 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.901990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb" event={"ID":"c5bceb52-bdd2-413e-8871-dcefa851d0d9","Type":"ContainerStarted","Data":"8e296c3c4258f2ec4c0e2eb0ff89d00b806621d14067f463b2caaa0b565aefb5"}
Jul 07 19:10:41.906205 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:41.906183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:42.002732 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:42.002705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:10:42.035150 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:42.035107    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"]
Jul 07 19:10:42.041497 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:42.041471    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a51f612_c192_4159_88ba_00234127b85d.slice/crio-fc4b0cc540afbb9bd97e9313bc4430c0490945d7aa885997b06cb3ccb29ae331 WatchSource:0}: Error finding container fc4b0cc540afbb9bd97e9313bc4430c0490945d7aa885997b06cb3ccb29ae331: Status 404 returned error can't find the container with id fc4b0cc540afbb9bd97e9313bc4430c0490945d7aa885997b06cb3ccb29ae331
Jul 07 19:10:42.127709 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:42.127635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xc9kj"]
Jul 07 19:10:42.130462 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:42.130415    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11398cfd_4692_40d0_bf1b_f84414c94184.slice/crio-f431d837ba858b3dc6bb027753ddf71173a1b36ab43d40cbe2c3b09823a75f45 WatchSource:0}: Error finding container f431d837ba858b3dc6bb027753ddf71173a1b36ab43d40cbe2c3b09823a75f45: Status 404 returned error can't find the container with id f431d837ba858b3dc6bb027753ddf71173a1b36ab43d40cbe2c3b09823a75f45
Jul 07 19:10:42.911015 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:42.910974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xc9kj" event={"ID":"11398cfd-4692-40d0-bf1b-f84414c94184","Type":"ContainerStarted","Data":"f431d837ba858b3dc6bb027753ddf71173a1b36ab43d40cbe2c3b09823a75f45"}
Jul 07 19:10:42.912580 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:42.912549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6" event={"ID":"8a51f612-c192-4159-88ba-00234127b85d","Type":"ContainerStarted","Data":"fc4b0cc540afbb9bd97e9313bc4430c0490945d7aa885997b06cb3ccb29ae331"}
Jul 07 19:10:44.781621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.781573    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:10:44.791865 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.791841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.795959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795278    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 19:10:44.795959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795464    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:10:44.795959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 19:10:44.795959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-85q68\""
Jul 07 19:10:44.795959 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795885    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 19:10:44.796185 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.795972    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 19:10:44.796185 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.796134    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 19:10:44.890495 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890425    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.890670 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.890670 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.890670 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.890802 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs88w\" (UniqueName: \"kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.890802 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.890761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991189 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991299 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991299 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991299 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs88w\" (UniqueName: \"kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991452 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.991452 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.991350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.992260 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.992233    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.992818 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.992776    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.992949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.992900    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.994507 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.994487    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:44.994672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:44.994653    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:45.002327 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.002305    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs88w\" (UniqueName: \"kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w\") pod \"console-f8f498b58-b2fh7\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") " pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:45.106707 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.106622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:45.266115 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.266073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:10:45.271374 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:45.271347    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae68fa8_9cc8_4356_9ec7_b9ce7c7bc682.slice/crio-2418d7489189c8685f9538afbb96a74790fa8c00f7b9caa26385d0a44a72645f WatchSource:0}: Error finding container 2418d7489189c8685f9538afbb96a74790fa8c00f7b9caa26385d0a44a72645f: Status 404 returned error can't find the container with id 2418d7489189c8685f9538afbb96a74790fa8c00f7b9caa26385d0a44a72645f
Jul 07 19:10:45.476273 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.476245    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"]
Jul 07 19:10:45.480624 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.480583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:45.482707 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.482681    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 07 19:10:45.482830 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.482731    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-c44ff\""
Jul 07 19:10:45.489242 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.489206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"]
Jul 07 19:10:45.597603 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.597544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d89b74ce-acc0-4fe2-867d-827281431cb6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jrn4n\" (UID: \"d89b74ce-acc0-4fe2-867d-827281431cb6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:45.698166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.698129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d89b74ce-acc0-4fe2-867d-827281431cb6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jrn4n\" (UID: \"d89b74ce-acc0-4fe2-867d-827281431cb6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:45.701115 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.701085    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d89b74ce-acc0-4fe2-867d-827281431cb6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jrn4n\" (UID: \"d89b74ce-acc0-4fe2-867d-827281431cb6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:45.793494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.793400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:45.926926 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.926881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb" event={"ID":"c5bceb52-bdd2-413e-8871-dcefa851d0d9","Type":"ContainerStarted","Data":"8d276efad92606dadadc1cbd86dfcc3900c9219b946156b83c4ae5926c05cb04"}
Jul 07 19:10:45.929704 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.929663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8f498b58-b2fh7" event={"ID":"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682","Type":"ContainerStarted","Data":"2418d7489189c8685f9538afbb96a74790fa8c00f7b9caa26385d0a44a72645f"}
Jul 07 19:10:45.946885 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:45.946809    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hw4cb" podStartSLOduration=33.395085938 podStartE2EDuration="36.946788374s" totalImagesPullingTime="3.551702436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:45.94594549 +0000 UTC m=+96.907396767" watchObservedRunningTime="2026-07-07 19:10:45.946788374 +0000 UTC m=+96.908239652"
Jul 07 19:10:46.758929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.758892    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:10:46.762437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.762393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.774647 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.774620    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:10:46.775052 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.775020    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq6bv\" (UniqueName: \"kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:46.908942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:46.908857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010003 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.009949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010110 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010110 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010110 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.010268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6bv\" (UniqueName: \"kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.011001 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010956    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.011001 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.010991    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.011292 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.011271    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.011357 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.011278    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.012566 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.012547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.012938 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.012919    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.020739 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.020717    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6bv\" (UniqueName: \"kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv\") pod \"console-8b6588454-gkr9p\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") " pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.065705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.065684    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"]
Jul 07 19:10:47.068050 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:47.068026    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89b74ce_acc0_4fe2_867d_827281431cb6.slice/crio-5922ea7bb1f5e5225b7400e5cc810b741b46e01a8e0e316504b371148c1f9d28 WatchSource:0}: Error finding container 5922ea7bb1f5e5225b7400e5cc810b741b46e01a8e0e316504b371148c1f9d28: Status 404 returned error can't find the container with id 5922ea7bb1f5e5225b7400e5cc810b741b46e01a8e0e316504b371148c1f9d28
Jul 07 19:10:47.076461 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.076440    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:47.210984 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.210941    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:10:47.215713 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:47.215686    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81694f13_9a20_4bac_bf58_9746e9b1d9bb.slice/crio-0db39edd687d83e5825d97614ab2dcabe14fa81dfb39d6d3808b3e0f8e198171 WatchSource:0}: Error finding container 0db39edd687d83e5825d97614ab2dcabe14fa81dfb39d6d3808b3e0f8e198171: Status 404 returned error can't find the container with id 0db39edd687d83e5825d97614ab2dcabe14fa81dfb39d6d3808b3e0f8e198171
Jul 07 19:10:47.940503 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.940462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6" event={"ID":"8a51f612-c192-4159-88ba-00234127b85d","Type":"ContainerStarted","Data":"5756ea1ebe5d358446e8d55602c4c25fa7c1409a31aea49e666ee7c71a742661"}
Jul 07 19:10:47.942787 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.942763    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:47.943856 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.943580    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6"
Jul 07 19:10:47.945972 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.945861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6588454-gkr9p" event={"ID":"81694f13-9a20-4bac-bf58-9746e9b1d9bb","Type":"ContainerStarted","Data":"0db39edd687d83e5825d97614ab2dcabe14fa81dfb39d6d3808b3e0f8e198171"}
Jul 07 19:10:47.947760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.947736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n" event={"ID":"d89b74ce-acc0-4fe2-867d-827281431cb6","Type":"ContainerStarted","Data":"5922ea7bb1f5e5225b7400e5cc810b741b46e01a8e0e316504b371148c1f9d28"}
Jul 07 19:10:47.960831 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:47.960725    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f985dc686-ldfg6" podStartSLOduration=1.996444135 podStartE2EDuration="6.960708132s" totalImagesPullingTime="4.964263997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:47.959123364 +0000 UTC m=+98.920574641" watchObservedRunningTime="2026-07-07 19:10:47.960708132 +0000 UTC m=+98.922159412"
Jul 07 19:10:48.954943 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:48.954909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n" event={"ID":"d89b74ce-acc0-4fe2-867d-827281431cb6","Type":"ContainerStarted","Data":"430cda3cb2538810d30c5560bc4aad483597f91ff8fe937c3ee84f82abed89c5"}
Jul 07 19:10:48.955515 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:48.955468    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:48.962721 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:48.962669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n"
Jul 07 19:10:48.977440 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:48.976951    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jrn4n" podStartSLOduration=2.685601039 podStartE2EDuration="3.976935222s" totalImagesPullingTime="1.291334183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:48.97575743 +0000 UTC m=+99.937208707" watchObservedRunningTime="2026-07-07 19:10:48.976935222 +0000 UTC m=+99.938386497"
Jul 07 19:10:49.330802 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.330769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:49.331046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.330844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:49.333488 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.333459    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae84ed10-ccc5-4a89-9374-bbbcda75246a-metrics-tls\") pod \"dns-default-whk7g\" (UID: \"ae84ed10-ccc5-4a89-9374-bbbcda75246a\") " pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:49.333616 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.333472    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"image-registry-9b6656699-7jr5k\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") " pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:49.597485 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.597374    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xhv4z\""
Jul 07 19:10:49.605562 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.605539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:49.625064 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.625035    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g92mm\""
Jul 07 19:10:49.633519 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:49.633494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:50.388638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.388608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:10:50.392927 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:50.392869    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b0f316_b64a_4015_8662_6fa3d051cff8.slice/crio-10f8191710389e24e48ef348aaa1ad9d490d4ca788d54e801c10e3e79e16ec6d WatchSource:0}: Error finding container 10f8191710389e24e48ef348aaa1ad9d490d4ca788d54e801c10e3e79e16ec6d: Status 404 returned error can't find the container with id 10f8191710389e24e48ef348aaa1ad9d490d4ca788d54e801c10e3e79e16ec6d
Jul 07 19:10:50.417119 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.417093    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-whk7g"]
Jul 07 19:10:50.422247 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:10:50.422200    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae84ed10_ccc5_4a89_9374_bbbcda75246a.slice/crio-8c5329631ebd71b290124756a6f6f1c83bb0f3e8a1f4438059c1a24c9bca007d WatchSource:0}: Error finding container 8c5329631ebd71b290124756a6f6f1c83bb0f3e8a1f4438059c1a24c9bca007d: Status 404 returned error can't find the container with id 8c5329631ebd71b290124756a6f6f1c83bb0f3e8a1f4438059c1a24c9bca007d
Jul 07 19:10:50.831155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.831125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n2tls"
Jul 07 19:10:50.965099 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.965066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whk7g" event={"ID":"ae84ed10-ccc5-4a89-9374-bbbcda75246a","Type":"ContainerStarted","Data":"8c5329631ebd71b290124756a6f6f1c83bb0f3e8a1f4438059c1a24c9bca007d"}
Jul 07 19:10:50.966512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.966475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8f498b58-b2fh7" event={"ID":"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682","Type":"ContainerStarted","Data":"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"}
Jul 07 19:10:50.967882 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.967859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" event={"ID":"06b0f316-b64a-4015-8662-6fa3d051cff8","Type":"ContainerStarted","Data":"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"}
Jul 07 19:10:50.967986 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.967893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" event={"ID":"06b0f316-b64a-4015-8662-6fa3d051cff8","Type":"ContainerStarted","Data":"10f8191710389e24e48ef348aaa1ad9d490d4ca788d54e801c10e3e79e16ec6d"}
Jul 07 19:10:50.967986 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.967924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:10:50.973272 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.973244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6588454-gkr9p" event={"ID":"81694f13-9a20-4bac-bf58-9746e9b1d9bb","Type":"ContainerStarted","Data":"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"}
Jul 07 19:10:50.985841 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:50.985790    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f8f498b58-b2fh7" podStartSLOduration=1.949539715 podStartE2EDuration="6.985774695s" totalImagesPullingTime="5.03623498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:50.985740275 +0000 UTC m=+101.947191553" watchObservedRunningTime="2026-07-07 19:10:50.985774695 +0000 UTC m=+101.947225973"
Jul 07 19:10:51.007814 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:51.007776    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" podStartSLOduration=102.007763639 podStartE2EDuration="1m42.007763639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:51.006445147 +0000 UTC m=+101.967896426" watchObservedRunningTime="2026-07-07 19:10:51.007763639 +0000 UTC m=+101.969214914"
Jul 07 19:10:51.025602 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:51.025544    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8b6588454-gkr9p" podStartSLOduration=1.916212653 podStartE2EDuration="5.025532429s" totalImagesPullingTime="3.109319776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:51.024114333 +0000 UTC m=+101.985565608" watchObservedRunningTime="2026-07-07 19:10:51.025532429 +0000 UTC m=+101.986983786"
Jul 07 19:10:53.988150 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:53.988117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whk7g" event={"ID":"ae84ed10-ccc5-4a89-9374-bbbcda75246a","Type":"ContainerStarted","Data":"434698241fe3ef543e41e398d5b40cd920b58cc16f1c5d525b1d4f7e7d8473bb"}
Jul 07 19:10:53.988626 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:53.988174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whk7g" event={"ID":"ae84ed10-ccc5-4a89-9374-bbbcda75246a","Type":"ContainerStarted","Data":"641ed5495b089a61fd7002e2039899d9f2781e96dc938be42d8d5c72e04201db"}
Jul 07 19:10:53.988626 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:53.988209    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-whk7g"
Jul 07 19:10:54.014533 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.014485    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-whk7g" podStartSLOduration=66.07787188 podStartE2EDuration="1m9.014457048s" totalImagesPullingTime="2.936585168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:10:54.013736329 +0000 UTC m=+104.975187607" watchObservedRunningTime="2026-07-07 19:10:54.014457048 +0000 UTC m=+104.975908324"
Jul 07 19:10:54.934191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.934156    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"]
Jul 07 19:10:54.937759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.937737    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jw24g"]
Jul 07 19:10:54.937903 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.937882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:54.939944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.939922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 07 19:10:54.940415 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.940396    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 07 19:10:54.941008 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.940991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s6v2k\""
Jul 07 19:10:54.941450 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.941282    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 19:10:54.941645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.941628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:54.944182 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.944164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 19:10:54.944333 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.944290    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 19:10:54.944446 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.944290    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 19:10:54.944446 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.944248    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wzk4r\""
Jul 07 19:10:54.949854 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.949834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"]
Jul 07 19:10:54.956519 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.956499    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"]
Jul 07 19:10:54.959852 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.959834    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:54.962521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.961768    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 07 19:10:54.962521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.962084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 07 19:10:54.962521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.962165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4qdk6\""
Jul 07 19:10:54.962521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.962254    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 07 19:10:54.972998 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:54.972977    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"]
Jul 07 19:10:55.084938 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.084908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-textfile\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.084978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zbwc\" (UniqueName: \"kubernetes.io/projected/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-api-access-8zbwc\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqp6\" (UniqueName: \"kubernetes.io/projected/2563567e-7226-4607-b9af-49b5d460b9eb-kube-api-access-vmqp6\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vbjm\" (UniqueName: \"kubernetes.io/projected/5ea69466-2fa0-43fb-8747-992942f1bc2b-kube-api-access-9vbjm\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-sys\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-root\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.085430 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2563567e-7226-4607-b9af-49b5d460b9eb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.086348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.086348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.086348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-wtmp\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.086348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085715    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-metrics-client-ca\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.086348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.085741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.106798 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.106772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:55.106912 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.106820    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:55.112442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.112413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:55.186740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.186740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.186740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-wtmp\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-metrics-client-ca\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-textfile\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.186842    2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbwc\" (UniqueName: \"kubernetes.io/projected/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-api-access-8zbwc\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186899    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-wtmp\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.186915    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls podName:ef5eea5f-b32a-4df8-8f0f-ea91359c93af nodeName:}" failed. No retries permitted until 2026-07-07 19:10:55.686891774 +0000 UTC m=+106.648343030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-bfx9p" (UID: "ef5eea5f-b32a-4df8-8f0f-ea91359c93af") : secret "kube-state-metrics-tls" not found
Jul 07 19:10:55.186996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.186975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqp6\" (UniqueName: \"kubernetes.io/projected/2563567e-7226-4607-b9af-49b5d460b9eb-kube-api-access-vmqp6\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vbjm\" (UniqueName: \"kubernetes.io/projected/5ea69466-2fa0-43fb-8747-992942f1bc2b-kube-api-access-9vbjm\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-sys\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-root\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2563567e-7226-4607-b9af-49b5d460b9eb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187335    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187454    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187459    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea69466-2fa0-43fb-8747-992942f1bc2b-metrics-client-ca\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.187514    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 19:10:55.187541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187517    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-root\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187552    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.187557    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea69466-2fa0-43fb-8747-992942f1bc2b-sys\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.187561    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls podName:5ea69466-2fa0-43fb-8747-992942f1bc2b nodeName:}" failed. No retries permitted until 2026-07-07 19:10:55.687546537 +0000 UTC m=+106.648997794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls") pod "node-exporter-jw24g" (UID: "5ea69466-2fa0-43fb-8747-992942f1bc2b") : secret "node-exporter-tls" not found
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.187645    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:10:55.187684    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls podName:2563567e-7226-4607-b9af-49b5d460b9eb nodeName:}" failed. No retries permitted until 2026-07-07 19:10:55.687670561 +0000 UTC m=+106.649121817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-b9lxh" (UID: "2563567e-7226-4607-b9af-49b5d460b9eb") : secret "openshift-state-metrics-tls" not found
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.188270    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2563567e-7226-4607-b9af-49b5d460b9eb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.188442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.188289    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.188910 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.188486    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-textfile\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.189982 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.189958    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.190081 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.190041    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.190296 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.190280    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.197383 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.197324    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vbjm\" (UniqueName: \"kubernetes.io/projected/5ea69466-2fa0-43fb-8747-992942f1bc2b-kube-api-access-9vbjm\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.198923 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.198896    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqp6\" (UniqueName: \"kubernetes.io/projected/2563567e-7226-4607-b9af-49b5d460b9eb-kube-api-access-vmqp6\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.199238 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.199213    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbwc\" (UniqueName: \"kubernetes.io/projected/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-api-access-8zbwc\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.693146 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.693107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.693319 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.693166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.693389 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.693344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.695907 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.695883    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ea69466-2fa0-43fb-8747-992942f1bc2b-node-exporter-tls\") pod \"node-exporter-jw24g\" (UID: \"5ea69466-2fa0-43fb-8747-992942f1bc2b\") " pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.696731 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.696707    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef5eea5f-b32a-4df8-8f0f-ea91359c93af-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bfx9p\" (UID: \"ef5eea5f-b32a-4df8-8f0f-ea91359c93af\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:55.696829 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.696707    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2563567e-7226-4607-b9af-49b5d460b9eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-b9lxh\" (UID: \"2563567e-7226-4607-b9af-49b5d460b9eb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.850825 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.850790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"
Jul 07 19:10:55.858215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.858192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jw24g"
Jul 07 19:10:55.869493 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:55.869471    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"
Jul 07 19:10:56.002530 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:56.002499    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:10:57.057807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.057773    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c7f79947-sdn5w"]
Jul 07 19:10:57.063255 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.063224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.067155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.066974    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hzk5l\""
Jul 07 19:10:57.067155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.066984    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 07 19:10:57.067155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.067007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 07 19:10:57.067155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.067069    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-brks9gl1mp21s\""
Jul 07 19:10:57.067442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.067190    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 07 19:10:57.067442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.067268    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 07 19:10:57.067442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.067317    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 07 19:10:57.076639 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.076617    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:57.076748 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.076664    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:57.079348 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.079323    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c7f79947-sdn5w"]
Jul 07 19:10:57.083033 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.083012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:57.206665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.206615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.206938 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.206874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srfm8\" (UniqueName: \"kubernetes.io/projected/b60535f9-a6c9-4414-9a5a-27316498bfae-kube-api-access-srfm8\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.206938 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.206934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.207093 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.206966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.207142 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.207111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.207277 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.207197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.207277 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.207227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b60535f9-a6c9-4414-9a5a-27316498bfae-metrics-client-ca\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.207277 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.207254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-grpc-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.307972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srfm8\" (UniqueName: \"kubernetes.io/projected/b60535f9-a6c9-4414-9a5a-27316498bfae-kube-api-access-srfm8\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308294 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308294 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308294 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308436 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308436 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b60535f9-a6c9-4414-9a5a-27316498bfae-metrics-client-ca\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.308436 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.308347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-grpc-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.309212 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.309167    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b60535f9-a6c9-4414-9a5a-27316498bfae-metrics-client-ca\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.311433 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.311294    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.311433 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.311335    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.311433 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.311339    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.311928 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.311910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-grpc-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.312154 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.312134    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-tls\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.312659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.312631    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b60535f9-a6c9-4414-9a5a-27316498bfae-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.326680 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.326229    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srfm8\" (UniqueName: \"kubernetes.io/projected/b60535f9-a6c9-4414-9a5a-27316498bfae-kube-api-access-srfm8\") pod \"thanos-querier-6c7f79947-sdn5w\" (UID: \"b60535f9-a6c9-4414-9a5a-27316498bfae\") " pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:57.375329 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:57.375302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:10:58.007288 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:58.007259    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:10:58.065729 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:10:58.065702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:11:00.165898 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.165863    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:11:00.174510 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.174470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.178661 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.178638    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:11:00.233997 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.233970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5r4h\" (UniqueName: \"kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234376 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.234376 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.234261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.334852 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5r4h\" (UniqueName: \"kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.334998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.335019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.335929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.335904    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.336100 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.336075    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.336172 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.336082    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.336172 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.336144    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.337938 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.337918    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.338175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.338154    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.346942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.346917    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5r4h\" (UniqueName: \"kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h\") pod \"console-77769d4944-5w9ng\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") " pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:00.485807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:00.485775    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:01.319183 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.319146    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 19:11:01.415989 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.415944    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 19:11:01.416156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.416086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.418745 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.418719    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2rpmx\""
Jul 07 19:11:01.418852 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.418719    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 19:11:01.419303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.418954    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 19:11:01.419303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.418962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 19:11:01.419303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.419081    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 19:11:01.419303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.419106    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 19:11:01.419303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.419207    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 19:11:01.419692 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.419659    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 19:11:01.419692 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.419672    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 19:11:01.420061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.420035    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 19:11:01.420149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.420109    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 19:11:01.420149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.420126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 19:11:01.420395 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.420379    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 19:11:01.420754 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.420733    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 19:11:01.421144 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.421114    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c3smvipf25s42\""
Jul 07 19:11:01.422255 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.422203    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 19:11:01.425687 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.425667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 19:11:01.443773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.443872 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.443872 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443841    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkrjh\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-kube-api-access-gkrjh\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.443960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.443960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.443993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.444614 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.444443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.544892 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.544863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.544932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.544965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.544982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkrjh\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-kube-api-access-gkrjh\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545843 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545843 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545843 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.545843 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.545444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.546435 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.546214    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.546571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.546451    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.548620 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.548437    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549152    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549320    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549661    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549662    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.549987    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.550177    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.550627    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-config\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.550960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.550936    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc7fcbc7-80cd-40e7-8061-4084348b43d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.551307 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.551284    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.551794 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.551769    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.552216 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.552189    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.552622 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.552574    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.553325 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.553298    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.553464 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.553445    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.554535 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.554517    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc7fcbc7-80cd-40e7-8061-4084348b43d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.564971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.564940    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkrjh\" (UniqueName: \"kubernetes.io/projected/bc7fcbc7-80cd-40e7-8061-4084348b43d4-kube-api-access-gkrjh\") pod \"prometheus-k8s-0\" (UID: \"bc7fcbc7-80cd-40e7-8061-4084348b43d4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:01.727993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:01.727963    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:02.502567 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:02.502546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:11:02.682253 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:11:02.682168    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268f2512_bba5_4af1_ba95_1440b3fb042d.slice/crio-273222d3166470994b9b5e6cd612dd257360d0c678f0e3d7c6f4e62dca3cfb0c WatchSource:0}: Error finding container 273222d3166470994b9b5e6cd612dd257360d0c678f0e3d7c6f4e62dca3cfb0c: Status 404 returned error can't find the container with id 273222d3166470994b9b5e6cd612dd257360d0c678f0e3d7c6f4e62dca3cfb0c
Jul 07 19:11:02.731859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:02.731830    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh"]
Jul 07 19:11:02.734314 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:02.734290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p"]
Jul 07 19:11:02.738830 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:02.738808    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c7f79947-sdn5w"]
Jul 07 19:11:02.740895 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:11:02.740765    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5eea5f_b32a_4df8_8f0f_ea91359c93af.slice/crio-55c4b726988fa17c86652101f75448501024d786d01cd2aabf69cf8750308d57 WatchSource:0}: Error finding container 55c4b726988fa17c86652101f75448501024d786d01cd2aabf69cf8750308d57: Status 404 returned error can't find the container with id 55c4b726988fa17c86652101f75448501024d786d01cd2aabf69cf8750308d57
Jul 07 19:11:02.743768 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:11:02.742747    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60535f9_a6c9_4414_9a5a_27316498bfae.slice/crio-861b839301504518c657bc9d75c296bda0e3a5119f97779e0ae6d241a9215866 WatchSource:0}: Error finding container 861b839301504518c657bc9d75c296bda0e3a5119f97779e0ae6d241a9215866: Status 404 returned error can't find the container with id 861b839301504518c657bc9d75c296bda0e3a5119f97779e0ae6d241a9215866
Jul 07 19:11:02.757781 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:02.757737    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 19:11:03.027627 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.027560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xc9kj" event={"ID":"11398cfd-4692-40d0-bf1b-f84414c94184","Type":"ContainerStarted","Data":"2ad73b37cac1609bacaf8a675fb344b2b5fa3c80539119159a660ea499798950"}
Jul 07 19:11:03.028524 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.028467    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:11:03.031830 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.031800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh" event={"ID":"2563567e-7226-4607-b9af-49b5d460b9eb","Type":"ContainerStarted","Data":"b69aed74769063ebf2ff90fc5a7d967d4fa55eff59da0c5f3e0608a976681e16"}
Jul 07 19:11:03.031960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.031863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh" event={"ID":"2563567e-7226-4607-b9af-49b5d460b9eb","Type":"ContainerStarted","Data":"9318038c73edc21121d1b87f742343b7493d796e16f44a946d47ac813ec1471e"}
Jul 07 19:11:03.031960 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.031886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh" event={"ID":"2563567e-7226-4607-b9af-49b5d460b9eb","Type":"ContainerStarted","Data":"ab8cbae002a6154fbcc00cc927c9780cbfe8fa400e937ec898f01e2e3f11e898"}
Jul 07 19:11:03.034692 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.034618    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"1d8dbf0398a4519e241bf7f0061f506f67e595eb9fbec9ccef5acc8bbf225070"}
Jul 07 19:11:03.036250 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.036145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p" event={"ID":"ef5eea5f-b32a-4df8-8f0f-ea91359c93af","Type":"ContainerStarted","Data":"55c4b726988fa17c86652101f75448501024d786d01cd2aabf69cf8750308d57"}
Jul 07 19:11:03.038172 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.038111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jw24g" event={"ID":"5ea69466-2fa0-43fb-8747-992942f1bc2b","Type":"ContainerStarted","Data":"ee45aed7e1e3cb257a22132503066521f8c14517de680303608c8dd3bcc2dd6b"}
Jul 07 19:11:03.039179 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.039128    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-xc9kj"
Jul 07 19:11:03.040187 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.040148    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77769d4944-5w9ng" event={"ID":"268f2512-bba5-4af1-ba95-1440b3fb042d","Type":"ContainerStarted","Data":"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"}
Jul 07 19:11:03.040282 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.040195    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77769d4944-5w9ng" event={"ID":"268f2512-bba5-4af1-ba95-1440b3fb042d","Type":"ContainerStarted","Data":"273222d3166470994b9b5e6cd612dd257360d0c678f0e3d7c6f4e62dca3cfb0c"}
Jul 07 19:11:03.042241 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.042218    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"861b839301504518c657bc9d75c296bda0e3a5119f97779e0ae6d241a9215866"}
Jul 07 19:11:03.057576 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.057522    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-xc9kj" podStartSLOduration=1.4331499509999999 podStartE2EDuration="22.057503049s" totalImagesPullingTime="20.624353098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:03.055257999 +0000 UTC m=+114.016709286" watchObservedRunningTime="2026-07-07 19:11:03.057503049 +0000 UTC m=+114.018954308"
Jul 07 19:11:03.995175 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:03.995139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-whk7g"
Jul 07 19:11:04.018904 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:04.018825    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77769d4944-5w9ng" podStartSLOduration=4.018801992 podStartE2EDuration="4.018801992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:03.126790325 +0000 UTC m=+114.088241601" watchObservedRunningTime="2026-07-07 19:11:04.018801992 +0000 UTC m=+114.980253271"
Jul 07 19:11:04.797145 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:04.797112    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:11:07.077777 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.076455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p" event={"ID":"ef5eea5f-b32a-4df8-8f0f-ea91359c93af","Type":"ContainerStarted","Data":"a85fa1d159351a6ad37988969d916d26af4f3ddc9336fc0ad30ac4cd8d80c186"}
Jul 07 19:11:07.077777 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.076527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p" event={"ID":"ef5eea5f-b32a-4df8-8f0f-ea91359c93af","Type":"ContainerStarted","Data":"22fc713bee8e35cad482565b910b3cb2e1745df9851763f749cdf3b61f9067fc"}
Jul 07 19:11:07.080482 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.080415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jw24g" event={"ID":"5ea69466-2fa0-43fb-8747-992942f1bc2b","Type":"ContainerStarted","Data":"a0a4f6e0cebdab10b659a09bd1542fce912e45b22cff84ecd42573ec4624e64b"}
Jul 07 19:11:07.083001 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.082902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"6350020c983657479de561f96e09fbd02fd84345a58fc14154781bd8bbc3ed46"}
Jul 07 19:11:07.088832 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.088805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh" event={"ID":"2563567e-7226-4607-b9af-49b5d460b9eb","Type":"ContainerStarted","Data":"c463dea51839aa6f74785ba4ccfd2cb0b5e5c4ca5b7626518af5b8ed49ba4423"}
Jul 07 19:11:07.092858 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.092061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"f4a0e49087cc80b064b8c1131b6e8dc0e7162ad36e6620d0f3ec7d7c44c91daf"}
Jul 07 19:11:07.201557 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:07.201090    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-b9lxh" podStartSLOduration=9.272022074 podStartE2EDuration="13.201067322s" totalImagesPullingTime="3.929045248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:07.19997302 +0000 UTC m=+118.161424298" watchObservedRunningTime="2026-07-07 19:11:07.201067322 +0000 UTC m=+118.162518598"
Jul 07 19:11:08.102712 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.102577    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc7fcbc7-80cd-40e7-8061-4084348b43d4" containerID="f4a0e49087cc80b064b8c1131b6e8dc0e7162ad36e6620d0f3ec7d7c44c91daf" exitCode=0
Jul 07 19:11:08.102712 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.102655    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerDied","Data":"f4a0e49087cc80b064b8c1131b6e8dc0e7162ad36e6620d0f3ec7d7c44c91daf"}
Jul 07 19:11:08.106743 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.105921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p" event={"ID":"ef5eea5f-b32a-4df8-8f0f-ea91359c93af","Type":"ContainerStarted","Data":"618f71213faee5ab71055720520fa5e0b7f66c1afaf4dc89515c36d021c62556"}
Jul 07 19:11:08.108537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.108513    2579 generic.go:356] "Generic (PLEG): container finished" podID="5ea69466-2fa0-43fb-8747-992942f1bc2b" containerID="a0a4f6e0cebdab10b659a09bd1542fce912e45b22cff84ecd42573ec4624e64b" exitCode=0
Jul 07 19:11:08.108792 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.108638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jw24g" event={"ID":"5ea69466-2fa0-43fb-8747-992942f1bc2b","Type":"ContainerDied","Data":"a0a4f6e0cebdab10b659a09bd1542fce912e45b22cff84ecd42573ec4624e64b"}
Jul 07 19:11:08.113366 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.113272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"ab00a14adf08b1a36032691d49f46f67ca44ef4b75da09a092b35538bc6c450c"}
Jul 07 19:11:08.113366 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.113331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"56b7179e10ec93f67093d5ee18aaea22a86135aa0b2eb6e395a12ed14cac69ab"}
Jul 07 19:11:08.125366 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:08.125303    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bfx9p" podStartSLOduration=10.076548792 podStartE2EDuration="14.125263409s" totalImagesPullingTime="4.048714617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:08.124737254 +0000 UTC m=+119.086188531" watchObservedRunningTime="2026-07-07 19:11:08.125263409 +0000 UTC m=+119.086714684"
Jul 07 19:11:09.123742 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:09.123168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jw24g" event={"ID":"5ea69466-2fa0-43fb-8747-992942f1bc2b","Type":"ContainerStarted","Data":"c38aca376949a56c5ff535f7ae072e5d06d7fa28e75be91ff58b1d717abd59ae"}
Jul 07 19:11:09.123742 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:09.123225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jw24g" event={"ID":"5ea69466-2fa0-43fb-8747-992942f1bc2b","Type":"ContainerStarted","Data":"23ccb526cd7115fd4d06562927ea74f248327c437dc9b991805314ef3eb94d7f"}
Jul 07 19:11:09.127551 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:09.127493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"23dfc7102aa55d4c94b201c8b46a545b48c5f1374e5d06e7801d2ef3fea3d286"}
Jul 07 19:11:09.147113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:09.147019    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jw24g" podStartSLOduration=10.552448373 podStartE2EDuration="15.147000423s" totalImagesPullingTime="4.421127375s" totalInitContainerRuntime="173.424675ms" isStatefulPod=true podCreationTimestamp="2026-07-07 19:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:09.14602611 +0000 UTC m=+120.107477391" watchObservedRunningTime="2026-07-07 19:11:09.147000423 +0000 UTC m=+120.108451699"
Jul 07 19:11:10.137162 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.137115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"5276444a18b37485e91ea651c537cb016b74f68bb476769b01759cb6ecc27066"}
Jul 07 19:11:10.137676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.137210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" event={"ID":"b60535f9-a6c9-4414-9a5a-27316498bfae","Type":"ContainerStarted","Data":"bf358adefeacd094710b23033182921c865b9d386af862e7542331dab9175605"}
Jul 07 19:11:10.163933 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.162812    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w" podStartSLOduration=7.566687932 podStartE2EDuration="13.162791703s" totalImagesPullingTime="5.596103771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:10:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:10.162006912 +0000 UTC m=+121.123458247" watchObservedRunningTime="2026-07-07 19:11:10.162791703 +0000 UTC m=+121.124242980"
Jul 07 19:11:10.487676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.486990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:10.487676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.487354    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:10.493008 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:10.492777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:11.143870 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:11.143774    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:11:11.150982 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:11.150952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:11:11.206942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:11.206905    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:11:12.152082 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:12.152054    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c7f79947-sdn5w"
Jul 07 19:11:13.153331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"f45e1859415a649500a8c5599f7c5f0d06e18cd70c685bf90e39e80c30f7fb03"}
Jul 07 19:11:13.153929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"ef7d7b137a137905e2c25f4d376e8aa90b386cfdd30e1fa0134b8da5e23d91de"}
Jul 07 19:11:13.153929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"d73ee0e27abc5b5193834cd32eb5952fff3502abd690a9996ff90e5d01c11651"}
Jul 07 19:11:13.153929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"795a6dbbb94395a9219a6c082b1ee797a3c225c5619ce4f95a981ed110248eeb"}
Jul 07 19:11:13.153929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"3385c99d687add3dd1d284ffb7ef95cae423c2b4b6e9eccd1b297194eb2f870a"}
Jul 07 19:11:13.153929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.153486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc7fcbc7-80cd-40e7-8061-4084348b43d4","Type":"ContainerStarted","Data":"626a43aa273debec0cd52e6f6075bbca3f5833e95613f91f1afd21cb956f17e3"}
Jul 07 19:11:13.186969 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:13.186918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.150392876 podStartE2EDuration="12.186897566s" totalImagesPullingTime="9.017033321s" totalInitContainerRuntime="19.471369ms" isStatefulPod=false podCreationTimestamp="2026-07-07 19:11:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:13.184617037 +0000 UTC m=+124.146068312" watchObservedRunningTime="2026-07-07 19:11:13.186897566 +0000 UTC m=+124.148348843"
Jul 07 19:11:14.802804 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:14.802775    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:11:16.728556 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:16.728528    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:11:19.231785 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:19.231744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:11:19.234248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:19.234226    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8237c9-6a76-49c4-b86c-818f95b3aa82-metrics-certs\") pod \"network-metrics-daemon-5xrb6\" (UID: \"dc8237c9-6a76-49c4-b86c-818f95b3aa82\") " pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:11:19.256360 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:19.256332    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jqhn7\""
Jul 07 19:11:19.265024 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:19.265003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xrb6"
Jul 07 19:11:19.431209 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:19.431190    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xrb6"]
Jul 07 19:11:19.433604 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:11:19.433556    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8237c9_6a76_49c4_b86c_818f95b3aa82.slice/crio-106b39c67da14ed661b168a637773c467743ab1eaa15a137d01f7644d81c1e06 WatchSource:0}: Error finding container 106b39c67da14ed661b168a637773c467743ab1eaa15a137d01f7644d81c1e06: Status 404 returned error can't find the container with id 106b39c67da14ed661b168a637773c467743ab1eaa15a137d01f7644d81c1e06
Jul 07 19:11:20.182506 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:20.182470    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xrb6" event={"ID":"dc8237c9-6a76-49c4-b86c-818f95b3aa82","Type":"ContainerStarted","Data":"106b39c67da14ed661b168a637773c467743ab1eaa15a137d01f7644d81c1e06"}
Jul 07 19:11:21.190264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:21.190223    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xrb6" event={"ID":"dc8237c9-6a76-49c4-b86c-818f95b3aa82","Type":"ContainerStarted","Data":"1f3ac4619f8042e055157819e4c833ebe139333fe3bc3ec5570f6d79414795e4"}
Jul 07 19:11:22.195859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:22.195825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xrb6" event={"ID":"dc8237c9-6a76-49c4-b86c-818f95b3aa82","Type":"ContainerStarted","Data":"f87493ccb8652473641ea7940a849353663932502c1bd91aef96212c3554e409"}
Jul 07 19:11:22.211834 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:22.211774    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5xrb6" podStartSLOduration=131.670504852 podStartE2EDuration="2m13.211731674s" totalImagesPullingTime="1.541226822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:11:22.211397966 +0000 UTC m=+133.172849236" watchObservedRunningTime="2026-07-07 19:11:22.211731674 +0000 UTC m=+133.173182950"
Jul 07 19:11:23.089996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.089936    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f8f498b58-b2fh7" podUID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" containerName="console" containerID="cri-o://48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93" gracePeriod=15
Jul 07 19:11:23.376265 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.376233    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:11:23.466191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466234    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466286    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs88w\" (UniqueName: \"kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466377    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466624 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466401    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config\") pod \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\" (UID: \"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682\") "
Jul 07 19:11:23.466739 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466710    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:23.466809 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466716    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config" (OuterVolumeSpecName: "console-config") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:23.466809 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.466782    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:23.468789 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.468762    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:23.468906 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.468826    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:23.468906 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.468876    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w" (OuterVolumeSpecName: "kube-api-access-zs88w") pod "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" (UID: "3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682"). InnerVolumeSpecName "kube-api-access-zs88w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:11:23.567623 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567579    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs88w\" (UniqueName: \"kubernetes.io/projected/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-kube-api-access-zs88w\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:23.567623 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567625    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-oauth-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:23.567790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567639    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-oauth-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:23.567790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567652    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:23.567790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567665    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:23.567790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:23.567677    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682-console-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:24.205268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.205230    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" containerID="48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93" exitCode=0
Jul 07 19:11:24.205504 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.205279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8f498b58-b2fh7" event={"ID":"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682","Type":"ContainerDied","Data":"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"}
Jul 07 19:11:24.205504 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.205327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8f498b58-b2fh7" event={"ID":"3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682","Type":"ContainerDied","Data":"2418d7489189c8685f9538afbb96a74790fa8c00f7b9caa26385d0a44a72645f"}
Jul 07 19:11:24.205504 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.205350    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8f498b58-b2fh7"
Jul 07 19:11:24.205504 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.205364    2579 scope.go:117] "RemoveContainer" containerID="48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"
Jul 07 19:11:24.215936 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.215913    2579 scope.go:117] "RemoveContainer" containerID="48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"
Jul 07 19:11:24.216226 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:11:24.216206    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93\": container with ID starting with 48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93 not found: ID does not exist" containerID="48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"
Jul 07 19:11:24.216281 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.216235    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93"} err="failed to get container status \"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93\": rpc error: code = NotFound desc = could not find container \"48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93\": container with ID starting with 48b7d88dcfbfae0f932f87d67630f9c27f6d150b6d58f0a1a2cd31cdf2745f93 not found: ID does not exist"
Jul 07 19:11:24.222646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.222622    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:11:24.226950 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:24.226929    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f8f498b58-b2fh7"]
Jul 07 19:11:25.547171 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:25.547138    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" path="/var/lib/kubelet/pods/3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682/volumes"
Jul 07 19:11:29.827680 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:29.827614    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" podUID="06b0f316-b64a-4015-8662-6fa3d051cff8" containerName="registry" containerID="cri-o://88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874" gracePeriod=30
Jul 07 19:11:30.098781 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.098727    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:11:30.231159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231127    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231209    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231258    2579 generic.go:356] "Generic (PLEG): container finished" podID="06b0f316-b64a-4015-8662-6fa3d051cff8" containerID="88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874" exitCode=0
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231268    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231310    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" event={"ID":"06b0f316-b64a-4015-8662-6fa3d051cff8","Type":"ContainerDied","Data":"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"}
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231350    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24bpx\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx\") pod \"06b0f316-b64a-4015-8662-6fa3d051cff8\" (UID: \"06b0f316-b64a-4015-8662-6fa3d051cff8\") "
Jul 07 19:11:30.231398 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231387    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b6656699-7jr5k" event={"ID":"06b0f316-b64a-4015-8662-6fa3d051cff8","Type":"ContainerDied","Data":"10f8191710389e24e48ef348aaa1ad9d490d4ca788d54e801c10e3e79e16ec6d"}
Jul 07 19:11:30.231866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231401    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b6656699-7jr5k"
Jul 07 19:11:30.231866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231414    2579 scope.go:117] "RemoveContainer" containerID="88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"
Jul 07 19:11:30.231958 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.231885    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:30.233017 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.232415    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:30.234069 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.234047    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:11:30.234215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.234190    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:11:30.234606 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.234539    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:30.234694 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.234607    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx" (OuterVolumeSpecName: "kube-api-access-24bpx") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "kube-api-access-24bpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:11:30.234749 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.234698    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:30.240428 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.240398    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "06b0f316-b64a-4015-8662-6fa3d051cff8" (UID: "06b0f316-b64a-4015-8662-6fa3d051cff8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:11:30.249527 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.249507    2579 scope.go:117] "RemoveContainer" containerID="88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"
Jul 07 19:11:30.249810 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:11:30.249788    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874\": container with ID starting with 88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874 not found: ID does not exist" containerID="88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"
Jul 07 19:11:30.249898 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.249816    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874"} err="failed to get container status \"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874\": rpc error: code = NotFound desc = could not find container \"88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874\": container with ID starting with 88b5b15abcb286d4ed8e6ffeee3f8707bbaf396d52d33720f565cd03b52b6874 not found: ID does not exist"
Jul 07 19:11:30.332949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332917    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-tls\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.332949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332943    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-installation-pull-secrets\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.332949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332951    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06b0f316-b64a-4015-8662-6fa3d051cff8-ca-trust-extracted\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.332949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332959    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-registry-certificates\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.333186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332967    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24bpx\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-kube-api-access-24bpx\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.333186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332975    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06b0f316-b64a-4015-8662-6fa3d051cff8-bound-sa-token\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.333186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332983    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06b0f316-b64a-4015-8662-6fa3d051cff8-trusted-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.333186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.332990    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06b0f316-b64a-4015-8662-6fa3d051cff8-image-registry-private-configuration\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:30.553707 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.553681    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:11:30.558030 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:30.558004    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9b6656699-7jr5k"]
Jul 07 19:11:31.547012 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:31.546979    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06b0f316-b64a-4015-8662-6fa3d051cff8" path="/var/lib/kubelet/pods/06b0f316-b64a-4015-8662-6fa3d051cff8/volumes"
Jul 07 19:11:36.240248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.240201    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8b6588454-gkr9p" podUID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" containerName="console" containerID="cri-o://2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae" gracePeriod=15
Jul 07 19:11:36.481462 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.481440    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:11:36.583095 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583013    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583095 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583095 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583073    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583095 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583087    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583435 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583435 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583151    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583435 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583188    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq6bv\" (UniqueName: \"kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv\") pod \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\" (UID: \"81694f13-9a20-4bac-bf58-9746e9b1d9bb\") "
Jul 07 19:11:36.583582 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583545    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:36.583676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583575    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:36.583676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583622    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config" (OuterVolumeSpecName: "console-config") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:36.583850 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.583831    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:11:36.585332 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.585311    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:36.585411 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.585396    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:11:36.585455 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.585412    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv" (OuterVolumeSpecName: "kube-api-access-vq6bv") pod "81694f13-9a20-4bac-bf58-9746e9b1d9bb" (UID: "81694f13-9a20-4bac-bf58-9746e9b1d9bb"). InnerVolumeSpecName "kube-api-access-vq6bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:11:36.684437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684414    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684434    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq6bv\" (UniqueName: \"kubernetes.io/projected/81694f13-9a20-4bac-bf58-9746e9b1d9bb-kube-api-access-vq6bv\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684443    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684451    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-oauth-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684461    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81694f13-9a20-4bac-bf58-9746e9b1d9bb-console-oauth-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684471    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-trusted-ca-bundle\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:36.684574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:36.684479    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81694f13-9a20-4bac-bf58-9746e9b1d9bb-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:11:37.260971 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.260934    2579 generic.go:356] "Generic (PLEG): container finished" podID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" containerID="2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae" exitCode=0
Jul 07 19:11:37.261396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.260977    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6588454-gkr9p"
Jul 07 19:11:37.261396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.260991    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6588454-gkr9p" event={"ID":"81694f13-9a20-4bac-bf58-9746e9b1d9bb","Type":"ContainerDied","Data":"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"}
Jul 07 19:11:37.261396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.261035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6588454-gkr9p" event={"ID":"81694f13-9a20-4bac-bf58-9746e9b1d9bb","Type":"ContainerDied","Data":"0db39edd687d83e5825d97614ab2dcabe14fa81dfb39d6d3808b3e0f8e198171"}
Jul 07 19:11:37.261396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.261058    2579 scope.go:117] "RemoveContainer" containerID="2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"
Jul 07 19:11:37.269722 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.269704    2579 scope.go:117] "RemoveContainer" containerID="2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"
Jul 07 19:11:37.270008 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:11:37.269991    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae\": container with ID starting with 2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae not found: ID does not exist" containerID="2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"
Jul 07 19:11:37.270061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.270017    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae"} err="failed to get container status \"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae\": rpc error: code = NotFound desc = could not find container \"2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae\": container with ID starting with 2cef92f8d25db3a4bf16a36b31b07ffe8222306e8bc3de2e984041da11fac0ae not found: ID does not exist"
Jul 07 19:11:37.282879 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.282852    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:11:37.287646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.287626    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8b6588454-gkr9p"]
Jul 07 19:11:37.547277 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:11:37.547201    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" path="/var/lib/kubelet/pods/81694f13-9a20-4bac-bf58-9746e9b1d9bb/volumes"
Jul 07 19:12:01.728804 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:01.728720    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:12:01.756651 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:01.756627    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:12:02.378669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:02.378643    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 19:12:23.816997 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.816957    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817420    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b0f316-b64a-4015-8662-6fa3d051cff8" containerName="registry"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817440    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b0f316-b64a-4015-8662-6fa3d051cff8" containerName="registry"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817458    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817467    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817494    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817503    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817578    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ae68fa8-9cc8-4356-9ec7-b9ce7c7bc682" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817611    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="81694f13-9a20-4bac-bf58-9746e9b1d9bb" containerName="console"
Jul 07 19:12:23.817638 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.817623    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b0f316-b64a-4015-8662-6fa3d051cff8" containerName="registry"
Jul 07 19:12:23.820644 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.820624    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.831416 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.831394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:12:23.955365 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phkcb\" (UniqueName: \"kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:23.955532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:23.955483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056226 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056226 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phkcb\" (UniqueName: \"kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.056457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.056407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.057126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.057058    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.057126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.057111    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.057322 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.057111    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.057322 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.057249    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.058896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.058878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.058951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.058903    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.067135 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.067088    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phkcb\" (UniqueName: \"kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb\") pod \"console-5db558b5b-hpsq9\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") " pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.129704 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.129676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:24.259484 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.259454    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:12:24.262020 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:12:24.261996    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf572c42_6872_47ba_aec5_0bec09e80766.slice/crio-ef0d421c019f5789bce04748732b8fc20330347fa833d20d18bca06629850e81 WatchSource:0}: Error finding container ef0d421c019f5789bce04748732b8fc20330347fa833d20d18bca06629850e81: Status 404 returned error can't find the container with id ef0d421c019f5789bce04748732b8fc20330347fa833d20d18bca06629850e81
Jul 07 19:12:24.443385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.443305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558b5b-hpsq9" event={"ID":"df572c42-6872-47ba-aec5-0bec09e80766","Type":"ContainerStarted","Data":"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"}
Jul 07 19:12:24.443385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.443354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558b5b-hpsq9" event={"ID":"df572c42-6872-47ba-aec5-0bec09e80766","Type":"ContainerStarted","Data":"ef0d421c019f5789bce04748732b8fc20330347fa833d20d18bca06629850e81"}
Jul 07 19:12:24.465459 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:24.465409    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db558b5b-hpsq9" podStartSLOduration=1.465391031 podStartE2EDuration="1.465391031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:12:24.464353241 +0000 UTC m=+195.425804530" watchObservedRunningTime="2026-07-07 19:12:24.465391031 +0000 UTC m=+195.426842307"
Jul 07 19:12:33.965678 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:33.965645    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:12:33.991080 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:33.991042    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:12:33.993266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:33.993243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.003850 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.003822    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:12:34.130695 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.130660    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:34.148115 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148253 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlmq7\" (UniqueName: \"kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148253 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148337 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148337 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148411 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.148411 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.148376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlmq7\" (UniqueName: \"kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249329 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249373 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249425 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249876 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249853    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249983 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249894    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.249983 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.249860    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.250335 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.250311    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.251949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.251920    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.252019 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.251956    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.259524 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.259501    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlmq7\" (UniqueName: \"kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7\") pod \"console-659b68478d-trz2v\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") " pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.304300 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.304278    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:34.436191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.436153    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:12:34.438975 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:12:34.438953    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ccf495_5e67_4fe4_b060_7bbf05aeb7f2.slice/crio-528badc5a6eb383b110f1b0fe05b8c0baef5da20784012f630d82cb6fc6932ca WatchSource:0}: Error finding container 528badc5a6eb383b110f1b0fe05b8c0baef5da20784012f630d82cb6fc6932ca: Status 404 returned error can't find the container with id 528badc5a6eb383b110f1b0fe05b8c0baef5da20784012f630d82cb6fc6932ca
Jul 07 19:12:34.483442 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:34.483417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659b68478d-trz2v" event={"ID":"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2","Type":"ContainerStarted","Data":"528badc5a6eb383b110f1b0fe05b8c0baef5da20784012f630d82cb6fc6932ca"}
Jul 07 19:12:35.489252 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:35.489213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659b68478d-trz2v" event={"ID":"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2","Type":"ContainerStarted","Data":"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"}
Jul 07 19:12:35.508266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:35.508215    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-659b68478d-trz2v" podStartSLOduration=2.508197627 podStartE2EDuration="2.508197627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:12:35.507101709 +0000 UTC m=+206.468552987" watchObservedRunningTime="2026-07-07 19:12:35.508197627 +0000 UTC m=+206.469648904"
Jul 07 19:12:44.305453 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:44.305412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:44.305846 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:44.305483    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:44.310280 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:44.310260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:44.529620 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:44.529579    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:12:44.584234 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:44.584154    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:12:58.986090 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:58.986028    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5db558b5b-hpsq9" podUID="df572c42-6872-47ba-aec5-0bec09e80766" containerName="console" containerID="cri-o://1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df" gracePeriod=15
Jul 07 19:12:59.260251 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.260226    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:59.359571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359540    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.359813 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359580    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phkcb\" (UniqueName: \"kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.359813 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359625    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.359813 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359641    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.359813 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.360046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359831    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.360046 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.359862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config\") pod \"df572c42-6872-47ba-aec5-0bec09e80766\" (UID: \"df572c42-6872-47ba-aec5-0bec09e80766\") "
Jul 07 19:12:59.360160 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.360043    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config" (OuterVolumeSpecName: "console-config") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:12:59.360160 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.360136    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:12:59.360256 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.360173    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca" (OuterVolumeSpecName: "service-ca") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:12:59.360256 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.360227    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:12:59.361992 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.361962    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb" (OuterVolumeSpecName: "kube-api-access-phkcb") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "kube-api-access-phkcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:12:59.361992 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.361966    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:12:59.362116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.362018    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df572c42-6872-47ba-aec5-0bec09e80766" (UID: "df572c42-6872-47ba-aec5-0bec09e80766"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:12:59.461029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.460996    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461031    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461042    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df572c42-6872-47ba-aec5-0bec09e80766-console-oauth-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461051    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-oauth-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461060    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phkcb\" (UniqueName: \"kubernetes.io/projected/df572c42-6872-47ba-aec5-0bec09e80766-kube-api-access-phkcb\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461069    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-trusted-ca-bundle\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.461266 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.461076    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df572c42-6872-47ba-aec5-0bec09e80766-console-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:12:59.582987 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.582954    2579 generic.go:356] "Generic (PLEG): container finished" podID="df572c42-6872-47ba-aec5-0bec09e80766" containerID="1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df" exitCode=0
Jul 07 19:12:59.583127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.583009    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558b5b-hpsq9"
Jul 07 19:12:59.583127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.583011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558b5b-hpsq9" event={"ID":"df572c42-6872-47ba-aec5-0bec09e80766","Type":"ContainerDied","Data":"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"}
Jul 07 19:12:59.583127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.583087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558b5b-hpsq9" event={"ID":"df572c42-6872-47ba-aec5-0bec09e80766","Type":"ContainerDied","Data":"ef0d421c019f5789bce04748732b8fc20330347fa833d20d18bca06629850e81"}
Jul 07 19:12:59.583127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.583115    2579 scope.go:117] "RemoveContainer" containerID="1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"
Jul 07 19:12:59.591259 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.591244    2579 scope.go:117] "RemoveContainer" containerID="1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"
Jul 07 19:12:59.591553 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:12:59.591536    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df\": container with ID starting with 1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df not found: ID does not exist" containerID="1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"
Jul 07 19:12:59.591660 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.591565    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df"} err="failed to get container status \"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df\": rpc error: code = NotFound desc = could not find container \"1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df\": container with ID starting with 1128c99334e6a7a43862defca6664797fb9cb89408a5dbd33f551c351f2624df not found: ID does not exist"
Jul 07 19:12:59.600767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.600746    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:12:59.605627 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:12:59.605609    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db558b5b-hpsq9"]
Jul 07 19:13:01.547407 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:01.547376    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df572c42-6872-47ba-aec5-0bec09e80766" path="/var/lib/kubelet/pods/df572c42-6872-47ba-aec5-0bec09e80766/volumes"
Jul 07 19:13:09.606088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.606049    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77769d4944-5w9ng" podUID="268f2512-bba5-4af1-ba95-1440b3fb042d" containerName="console" containerID="cri-o://82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe" gracePeriod=15
Jul 07 19:13:09.846181 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.846153    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:13:09.946537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946501    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.946782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946565    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.946782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.946782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.946782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.946782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946768    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5r4h\" (UniqueName: \"kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.947047 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946793    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca\") pod \"268f2512-bba5-4af1-ba95-1440b3fb042d\" (UID: \"268f2512-bba5-4af1-ba95-1440b3fb042d\") "
Jul 07 19:13:09.947047 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.946954    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:13:09.947250 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.947226    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config" (OuterVolumeSpecName: "console-config") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:13:09.947311 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.947245    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:13:09.947311 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.947276    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca" (OuterVolumeSpecName: "service-ca") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:13:09.949039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.949018    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:13:09.949138 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.949035    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h" (OuterVolumeSpecName: "kube-api-access-r5r4h") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "kube-api-access-r5r4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:13:09.949138 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:09.949041    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "268f2512-bba5-4af1-ba95-1440b3fb042d" (UID: "268f2512-bba5-4af1-ba95-1440b3fb042d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047440    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-oauth-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047465    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-trusted-ca-bundle\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047475    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5r4h\" (UniqueName: \"kubernetes.io/projected/268f2512-bba5-4af1-ba95-1440b3fb042d-kube-api-access-r5r4h\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047483    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047492    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-oauth-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047500    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/268f2512-bba5-4af1-ba95-1440b3fb042d-console-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.047516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.047508    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/268f2512-bba5-4af1-ba95-1440b3fb042d-console-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:13:10.627077 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.627042    2579 generic.go:356] "Generic (PLEG): container finished" podID="268f2512-bba5-4af1-ba95-1440b3fb042d" containerID="82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe" exitCode=0
Jul 07 19:13:10.627554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.627101    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77769d4944-5w9ng"
Jul 07 19:13:10.627554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.627121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77769d4944-5w9ng" event={"ID":"268f2512-bba5-4af1-ba95-1440b3fb042d","Type":"ContainerDied","Data":"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"}
Jul 07 19:13:10.627554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.627160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77769d4944-5w9ng" event={"ID":"268f2512-bba5-4af1-ba95-1440b3fb042d","Type":"ContainerDied","Data":"273222d3166470994b9b5e6cd612dd257360d0c678f0e3d7c6f4e62dca3cfb0c"}
Jul 07 19:13:10.627554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.627179    2579 scope.go:117] "RemoveContainer" containerID="82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"
Jul 07 19:13:10.635740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.635726    2579 scope.go:117] "RemoveContainer" containerID="82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"
Jul 07 19:13:10.636012 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:13:10.635993    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe\": container with ID starting with 82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe not found: ID does not exist" containerID="82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"
Jul 07 19:13:10.636057 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.636020    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe"} err="failed to get container status \"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe\": rpc error: code = NotFound desc = could not find container \"82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe\": container with ID starting with 82ef93d5f0ed763bb58689a84c13b12f9df8b1579026a35f4b7d342bd3e72ebe not found: ID does not exist"
Jul 07 19:13:10.651907 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.651882    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:13:10.657787 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:10.657769    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77769d4944-5w9ng"]
Jul 07 19:13:11.551070 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:13:11.551037    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="268f2512-bba5-4af1-ba95-1440b3fb042d" path="/var/lib/kubelet/pods/268f2512-bba5-4af1-ba95-1440b3fb042d/volumes"
Jul 07 19:14:09.370948 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:09.370920    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 19:14:27.196509 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196479    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jfqwn"]
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196815    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="268f2512-bba5-4af1-ba95-1440b3fb042d" containerName="console"
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196826    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="268f2512-bba5-4af1-ba95-1440b3fb042d" containerName="console"
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196836    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df572c42-6872-47ba-aec5-0bec09e80766" containerName="console"
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196842    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="df572c42-6872-47ba-aec5-0bec09e80766" containerName="console"
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196888    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="268f2512-bba5-4af1-ba95-1440b3fb042d" containerName="console"
Jul 07 19:14:27.198944 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.196898    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="df572c42-6872-47ba-aec5-0bec09e80766" containerName="console"
Jul 07 19:14:27.199790 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.199775    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.201566 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.201548    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4lwnm\""
Jul 07 19:14:27.201688 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.201581    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 19:14:27.201958 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.201944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 19:14:27.210297 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.210274    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jfqwn"]
Jul 07 19:14:27.298873 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.298842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmck9\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-kube-api-access-jmck9\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.299007 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.298900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.399646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.399610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmck9\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-kube-api-access-jmck9\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.399788 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.399704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.409951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.409927    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.410054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.409934    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmck9\" (UniqueName: \"kubernetes.io/projected/a5e36a28-4e7e-4fee-8fc7-dd9632a086a4-kube-api-access-jmck9\") pod \"cert-manager-55b4d954d5-jfqwn\" (UID: \"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4\") " pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.519607 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.519569    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jfqwn"
Jul 07 19:14:27.646418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.646396    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jfqwn"]
Jul 07 19:14:27.648998 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:14:27.648969    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5e36a28_4e7e_4fee_8fc7_dd9632a086a4.slice/crio-a4aaec4680766dcfa21fdb70aa6263215d0e165e07c839c0178faaa5278f2092 WatchSource:0}: Error finding container a4aaec4680766dcfa21fdb70aa6263215d0e165e07c839c0178faaa5278f2092: Status 404 returned error can't find the container with id a4aaec4680766dcfa21fdb70aa6263215d0e165e07c839c0178faaa5278f2092
Jul 07 19:14:27.651005 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.650981    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 19:14:27.928416 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:27.928333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jfqwn" event={"ID":"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4","Type":"ContainerStarted","Data":"a4aaec4680766dcfa21fdb70aa6263215d0e165e07c839c0178faaa5278f2092"}
Jul 07 19:14:30.943460 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:30.943381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jfqwn" event={"ID":"a5e36a28-4e7e-4fee-8fc7-dd9632a086a4","Type":"ContainerStarted","Data":"247ae6e8fbdb3d35c5394f58ac01ba513d4f77346537d81d707cf1f04491aa3b"}
Jul 07 19:14:30.961613 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:30.961557    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jfqwn" podStartSLOduration=1.102578796 podStartE2EDuration="3.961542243s" totalImagesPullingTime="2.858963447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:14:30.960513324 +0000 UTC m=+321.921964600" watchObservedRunningTime="2026-07-07 19:14:30.961542243 +0000 UTC m=+321.922993521"
Jul 07 19:14:51.807026 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.805962    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"]
Jul 07 19:14:51.810705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.810681    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.819122 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.819100    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2ttzd\""
Jul 07 19:14:51.819236 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.819140    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 19:14:51.819298 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.819232    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 19:14:51.819400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.819385    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 19:14:51.821045 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.821031    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 19:14:51.823379 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.823352    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"]
Jul 07 19:14:51.886930 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.886895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-webhook-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.887106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.886937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjjv8\" (UniqueName: \"kubernetes.io/projected/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-kube-api-access-bjjv8\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.887106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.887003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-manifests\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.887106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.887031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.887106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.887073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-tmp\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988197 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-webhook-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjjv8\" (UniqueName: \"kubernetes.io/projected/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-kube-api-access-bjjv8\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-manifests\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-tmp\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.988726 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.988700    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-manifests\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.990735 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.990700    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-tmp\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.990834 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.990785    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-webhook-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.990889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.990826    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:51.998353 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:51.998330    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjjv8\" (UniqueName: \"kubernetes.io/projected/1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64-kube-api-access-bjjv8\") pod \"opendatahub-operator-controller-manager-75847b84df-w4fd4\" (UID: \"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64\") " pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:52.122730 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:52.122651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:52.279814 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:52.279733    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"]
Jul 07 19:14:52.283122 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:14:52.283093    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7d35c7_bcbf_4c1d_90aa_f54e9c35be64.slice/crio-fdd212d7e00934ee28f4e39bcbea53ca387ccc3436036d30da9301b1ee82bb3b WatchSource:0}: Error finding container fdd212d7e00934ee28f4e39bcbea53ca387ccc3436036d30da9301b1ee82bb3b: Status 404 returned error can't find the container with id fdd212d7e00934ee28f4e39bcbea53ca387ccc3436036d30da9301b1ee82bb3b
Jul 07 19:14:53.030390 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:53.030319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4" event={"ID":"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64","Type":"ContainerStarted","Data":"fdd212d7e00934ee28f4e39bcbea53ca387ccc3436036d30da9301b1ee82bb3b"}
Jul 07 19:14:56.045437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:56.045402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4" event={"ID":"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64","Type":"ContainerStarted","Data":"f34fff98fc120aa2125c57a3fe576de74202301e13d311749f0f5658b27fa984"}
Jul 07 19:14:57.051102 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:57.051069    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64" containerID="f34fff98fc120aa2125c57a3fe576de74202301e13d311749f0f5658b27fa984" exitCode=0
Jul 07 19:14:57.051493 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:57.051155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4" event={"ID":"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64","Type":"ContainerDied","Data":"f34fff98fc120aa2125c57a3fe576de74202301e13d311749f0f5658b27fa984"}
Jul 07 19:14:58.057296 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:58.057260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4" event={"ID":"1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64","Type":"ContainerStarted","Data":"e037632788ce294e65f1a03d2b285083b2b7475ab83b347846c6604dcc9f2c2f"}
Jul 07 19:14:58.057793 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:58.057356    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:14:58.083474 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:14:58.083422    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4" podStartSLOduration=2.675961173 podStartE2EDuration="7.083406048s" totalImagesPullingTime="3.396726747s" totalInitContainerRuntime="1.010718128s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:14:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:14:58.081159953 +0000 UTC m=+349.042611209" watchObservedRunningTime="2026-07-07 19:14:58.083406048 +0000 UTC m=+349.044857403"
Jul 07 19:15:09.063935 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:09.063864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75847b84df-w4fd4"
Jul 07 19:15:10.628411 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.628378    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"]
Jul 07 19:15:10.636800 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.636775    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.639283 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639260    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 19:15:10.639471 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639446    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mfdtv\""
Jul 07 19:15:10.639579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639481    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 19:15:10.639579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639451    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 19:15:10.639579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639508    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 19:15:10.639579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.639492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 19:15:10.648233 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.648211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"]
Jul 07 19:15:10.744021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.743985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb6dff71-aa05-4b75-9511-9d86c82b05a7-manager-config\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.744021 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.744021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-metrics-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.744283 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.744051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmj5v\" (UniqueName: \"kubernetes.io/projected/cb6dff71-aa05-4b75-9511-9d86c82b05a7-kube-api-access-tmj5v\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.744283 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.744131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.845248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.845204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.845450 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.845292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb6dff71-aa05-4b75-9511-9d86c82b05a7-manager-config\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.845450 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.845312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-metrics-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.845450 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.845336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmj5v\" (UniqueName: \"kubernetes.io/projected/cb6dff71-aa05-4b75-9511-9d86c82b05a7-kube-api-access-tmj5v\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.846170 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.846091    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb6dff71-aa05-4b75-9511-9d86c82b05a7-manager-config\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.847972 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.847947    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.848055 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.848008    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb6dff71-aa05-4b75-9511-9d86c82b05a7-metrics-cert\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.859657 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.859629    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmj5v\" (UniqueName: \"kubernetes.io/projected/cb6dff71-aa05-4b75-9511-9d86c82b05a7-kube-api-access-tmj5v\") pod \"lws-controller-manager-76668d465c-g7jbt\" (UID: \"cb6dff71-aa05-4b75-9511-9d86c82b05a7\") " pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:10.945889 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:10.945806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:11.113005 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.112971    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"]
Jul 07 19:15:11.117969 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:15:11.117940    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6dff71_aa05_4b75_9511_9d86c82b05a7.slice/crio-34d5a1e82544b61e1f5b516dd7703ff7bf33a8db0085043dec834efb0ec561a6 WatchSource:0}: Error finding container 34d5a1e82544b61e1f5b516dd7703ff7bf33a8db0085043dec834efb0ec561a6: Status 404 returned error can't find the container with id 34d5a1e82544b61e1f5b516dd7703ff7bf33a8db0085043dec834efb0ec561a6
Jul 07 19:15:11.746043 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.744386    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"]
Jul 07 19:15:11.748252 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.748230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.751400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.750835    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 19:15:11.751400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.750841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 19:15:11.751400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.750981    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-29rlv\""
Jul 07 19:15:11.751400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.751215    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 19:15:11.751400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.751233    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 19:15:11.764125 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.764060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"]
Jul 07 19:15:11.854053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.854006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjmlf\" (UniqueName: \"kubernetes.io/projected/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-kube-api-access-rjmlf\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.854239 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.854189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.854239 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.854229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tmp\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.955096 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.955054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.955096 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.955102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tmp\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.955336 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.955179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjmlf\" (UniqueName: \"kubernetes.io/projected/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-kube-api-access-rjmlf\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.955336 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:15:11.955223    2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 07 19:15:11.955336 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:15:11.955293    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs podName:6230ef1f-26c1-4e0d-8988-9da05bb35cc0 nodeName:}" failed. No retries permitted until 2026-07-07 19:15:12.455272629 +0000 UTC m=+363.416723900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs") pod "kube-auth-proxy-598d7cb498-b2rmr" (UID: "6230ef1f-26c1-4e0d-8988-9da05bb35cc0") : secret "kube-auth-proxy-tls" not found
Jul 07 19:15:11.958084 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.958050    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tmp\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:11.970149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:11.970106    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjmlf\" (UniqueName: \"kubernetes.io/projected/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-kube-api-access-rjmlf\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:12.117901 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:12.117810    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt" event={"ID":"cb6dff71-aa05-4b75-9511-9d86c82b05a7","Type":"ContainerStarted","Data":"34d5a1e82544b61e1f5b516dd7703ff7bf33a8db0085043dec834efb0ec561a6"}
Jul 07 19:15:12.459405 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:12.459297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:12.462210 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:12.462183    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6230ef1f-26c1-4e0d-8988-9da05bb35cc0-tls-certs\") pod \"kube-auth-proxy-598d7cb498-b2rmr\" (UID: \"6230ef1f-26c1-4e0d-8988-9da05bb35cc0\") " pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:12.662030 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:12.662000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"
Jul 07 19:15:13.400503 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:13.400478    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr"]
Jul 07 19:15:13.402069 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:15:13.402036    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6230ef1f_26c1_4e0d_8988_9da05bb35cc0.slice/crio-23a43579e96d87bd7faf0bc72fc0820312667bdf3326994b0ab69227be150a41 WatchSource:0}: Error finding container 23a43579e96d87bd7faf0bc72fc0820312667bdf3326994b0ab69227be150a41: Status 404 returned error can't find the container with id 23a43579e96d87bd7faf0bc72fc0820312667bdf3326994b0ab69227be150a41
Jul 07 19:15:14.130332 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:14.130280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt" event={"ID":"cb6dff71-aa05-4b75-9511-9d86c82b05a7","Type":"ContainerStarted","Data":"392eab5815ce656b2582d4ef7b436b60a0d9d8090dad166ccaabb36ac42ba6df"}
Jul 07 19:15:14.130534 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:14.130392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:15:14.131637 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:14.131607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr" event={"ID":"6230ef1f-26c1-4e0d-8988-9da05bb35cc0","Type":"ContainerStarted","Data":"23a43579e96d87bd7faf0bc72fc0820312667bdf3326994b0ab69227be150a41"}
Jul 07 19:15:14.148633 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:14.148561    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt" podStartSLOduration=1.9696343189999999 podStartE2EDuration="4.14854298s" totalImagesPullingTime="2.178908661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:15:14.146419846 +0000 UTC m=+365.107871121" watchObservedRunningTime="2026-07-07 19:15:14.14854298 +0000 UTC m=+365.109994255"
Jul 07 19:15:16.140572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:16.140537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr" event={"ID":"6230ef1f-26c1-4e0d-8988-9da05bb35cc0","Type":"ContainerStarted","Data":"25756081bd06ca7a952722ebdce000241c7da55beeef2d69284d9da5f8242d6d"}
Jul 07 19:15:16.168994 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:16.168944    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-598d7cb498-b2rmr" podStartSLOduration=2.632324433 podStartE2EDuration="5.168927827s" totalImagesPullingTime="2.536603394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:15:16.167155623 +0000 UTC m=+367.128606901" watchObservedRunningTime="2026-07-07 19:15:16.168927827 +0000 UTC m=+367.130379103"
Jul 07 19:15:25.138056 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:15:25.138026    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76668d465c-g7jbt"
Jul 07 19:16:56.320886 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.320857    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"]
Jul 07 19:16:56.324309 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.324289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.327619 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.327580    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 19:16:56.328664 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.328640    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 19:16:56.328762 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.328738    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lcbmm\""
Jul 07 19:16:56.328924 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.328912    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 19:16:56.328968 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.328941    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 19:16:56.340846 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.340823    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"]
Jul 07 19:16:56.434844 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.434808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5hzh\" (UniqueName: \"kubernetes.io/projected/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-kube-api-access-f5hzh\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.435018 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.434889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.435018 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.434931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.536351 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.536319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.536479 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.536359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.536479 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.536422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5hzh\" (UniqueName: \"kubernetes.io/projected/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-kube-api-access-f5hzh\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.537042 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.537022    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.538900 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.538881    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.546995 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.546971    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5hzh\" (UniqueName: \"kubernetes.io/projected/3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7-kube-api-access-f5hzh\") pod \"kuadrant-console-plugin-66b5774985-gxrzx\" (UID: \"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.633965 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.633889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"
Jul 07 19:16:56.774746 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:56.774605    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx"]
Jul 07 19:16:56.776935 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:16:56.776907    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b6b2ca7_8b88_4299_a2b1_86cbd2d954c7.slice/crio-099c727648d1e8aaa88d4e2ac77afc0659ebd3a59f3199bd1aa1c0f4bdaf8d90 WatchSource:0}: Error finding container 099c727648d1e8aaa88d4e2ac77afc0659ebd3a59f3199bd1aa1c0f4bdaf8d90: Status 404 returned error can't find the container with id 099c727648d1e8aaa88d4e2ac77afc0659ebd3a59f3199bd1aa1c0f4bdaf8d90
Jul 07 19:16:57.589867 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:16:57.589824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx" event={"ID":"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7","Type":"ContainerStarted","Data":"099c727648d1e8aaa88d4e2ac77afc0659ebd3a59f3199bd1aa1c0f4bdaf8d90"}
Jul 07 19:17:07.657546 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.657509    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:17:07.695563 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.695519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:17:07.695766 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.695661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.697842 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.697821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q5b5p\""
Jul 07 19:17:07.842056 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.842016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhsfz\" (UniqueName: \"kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.842056 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.842058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.942716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.942636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhsfz\" (UniqueName: \"kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.942716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.942671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.943023 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.943003    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:07.954341 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:07.954314    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhsfz\" (UniqueName: \"kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz\") pod \"kuadrant-operator-controller-manager-86878c769f-jxsll\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:08.012418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:08.012366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:18.586924 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:18.586903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:17:18.590193 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:17:18.590170    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a89d86_69df_45c5_8527_4c02526cee8b.slice/crio-63cef05f83af3b9c2b268122bfe954ede4b17111fafb158f7bd2257f1e5558a1 WatchSource:0}: Error finding container 63cef05f83af3b9c2b268122bfe954ede4b17111fafb158f7bd2257f1e5558a1: Status 404 returned error can't find the container with id 63cef05f83af3b9c2b268122bfe954ede4b17111fafb158f7bd2257f1e5558a1
Jul 07 19:17:18.699126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:18.699041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx" event={"ID":"3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7","Type":"ContainerStarted","Data":"949971a3f65d3f91e938860d4ebf24a81c326649f9dcad7b49cfbe83e6672d18"}
Jul 07 19:17:18.700355 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:18.700333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" event={"ID":"a2a89d86-69df-45c5-8527-4c02526cee8b","Type":"ContainerStarted","Data":"63cef05f83af3b9c2b268122bfe954ede4b17111fafb158f7bd2257f1e5558a1"}
Jul 07 19:17:18.722495 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:18.722452    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gxrzx" podStartSLOduration=0.961824783 podStartE2EDuration="22.722437069s" totalImagesPullingTime="21.760612286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:17:18.721198815 +0000 UTC m=+489.682650093" watchObservedRunningTime="2026-07-07 19:17:18.722437069 +0000 UTC m=+489.683888346"
Jul 07 19:17:24.257292 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.257247    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dcb5584c5-4zngf"]
Jul 07 19:17:24.288340 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.288302    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcb5584c5-4zngf"]
Jul 07 19:17:24.288516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.288477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396604 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-trusted-ca-bundle\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396776 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-service-ca\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396776 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-oauth-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396776 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-oauth-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396921 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrhmd\" (UniqueName: \"kubernetes.io/projected/04891dbd-6f31-4bd5-942d-9da5ec22616a-kube-api-access-hrhmd\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396921 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.396921 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.396915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.497926 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.497886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-service-ca\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.497944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-oauth-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.497970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-oauth-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrhmd\" (UniqueName: \"kubernetes.io/projected/04891dbd-6f31-4bd5-942d-9da5ec22616a-kube-api-access-hrhmd\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498126 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498342 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.498342 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-trusted-ca-bundle\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.499037 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498765    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-service-ca\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.499037 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.498932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.499303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.499279    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-trusted-ca-bundle\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.499303 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.499296    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04891dbd-6f31-4bd5-942d-9da5ec22616a-oauth-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.500949 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.500926    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-oauth-config\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.501203 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.501182    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04891dbd-6f31-4bd5-942d-9da5ec22616a-console-serving-cert\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.507927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.507858    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrhmd\" (UniqueName: \"kubernetes.io/projected/04891dbd-6f31-4bd5-942d-9da5ec22616a-kube-api-access-hrhmd\") pod \"console-7dcb5584c5-4zngf\" (UID: \"04891dbd-6f31-4bd5-942d-9da5ec22616a\") " pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:24.600217 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:24.600179    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:25.053016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.052929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcb5584c5-4zngf"]
Jul 07 19:17:25.091012 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:17:25.090972    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04891dbd_6f31_4bd5_942d_9da5ec22616a.slice/crio-45bf125923133994eef7ffb8347e582d9ff43b1ea7005fa9e76826b65e4bf1cf WatchSource:0}: Error finding container 45bf125923133994eef7ffb8347e582d9ff43b1ea7005fa9e76826b65e4bf1cf: Status 404 returned error can't find the container with id 45bf125923133994eef7ffb8347e582d9ff43b1ea7005fa9e76826b65e4bf1cf
Jul 07 19:17:25.739814 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.739762    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" event={"ID":"a2a89d86-69df-45c5-8527-4c02526cee8b","Type":"ContainerStarted","Data":"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"}
Jul 07 19:17:25.740280 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.740082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:25.742751 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.742722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb5584c5-4zngf" event={"ID":"04891dbd-6f31-4bd5-942d-9da5ec22616a","Type":"ContainerStarted","Data":"55970bf945e90685dfb9dd3c0da7fcf50e0530ae1e73146e66b9bcbf1d5d796a"}
Jul 07 19:17:25.742993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.742764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb5584c5-4zngf" event={"ID":"04891dbd-6f31-4bd5-942d-9da5ec22616a","Type":"ContainerStarted","Data":"45bf125923133994eef7ffb8347e582d9ff43b1ea7005fa9e76826b65e4bf1cf"}
Jul 07 19:17:25.760315 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.760248    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" podStartSLOduration=12.243619532 podStartE2EDuration="18.760226126s" totalImagesPullingTime="6.516606594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:17:25.758739755 +0000 UTC m=+496.720191034" watchObservedRunningTime="2026-07-07 19:17:25.760226126 +0000 UTC m=+496.721677404"
Jul 07 19:17:25.779745 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:25.779678    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dcb5584c5-4zngf" podStartSLOduration=1.779659744 podStartE2EDuration="1.779659744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:17:25.778005852 +0000 UTC m=+496.739457131" watchObservedRunningTime="2026-07-07 19:17:25.779659744 +0000 UTC m=+496.741111020"
Jul 07 19:17:34.600378 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:34.600343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:34.600843 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:34.600407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:34.604622 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:34.604582    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:34.788439 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:34.788410    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dcb5584c5-4zngf"
Jul 07 19:17:34.847943 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:34.847910    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:17:36.749782 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:36.749752    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:17:56.502580 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:56.502545    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:17:56.823400 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:56.823305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:17:56.823548 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:56.823459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:17:56.825496 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:56.825477    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nvfkw\""
Jul 07 19:17:56.985357 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:56.985316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbw4m\" (UniqueName: \"kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m\") pod \"authorino-58cd955675-b9jsw\" (UID: \"73a69ec9-8617-4360-a8c3-d903e47f4bce\") " pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:17:57.086634 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:57.086533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbw4m\" (UniqueName: \"kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m\") pod \"authorino-58cd955675-b9jsw\" (UID: \"73a69ec9-8617-4360-a8c3-d903e47f4bce\") " pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:17:57.097060 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:57.097033    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbw4m\" (UniqueName: \"kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m\") pod \"authorino-58cd955675-b9jsw\" (UID: \"73a69ec9-8617-4360-a8c3-d903e47f4bce\") " pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:17:57.133872 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:57.133848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:17:57.286743 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:57.286718    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:17:57.288848 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:17:57.288817    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73a69ec9_8617_4360_a8c3_d903e47f4bce.slice/crio-7562f3f90aaa285afd527e759fcfb544f2e16d88948456844b55bcc4dc855372 WatchSource:0}: Error finding container 7562f3f90aaa285afd527e759fcfb544f2e16d88948456844b55bcc4dc855372: Status 404 returned error can't find the container with id 7562f3f90aaa285afd527e759fcfb544f2e16d88948456844b55bcc4dc855372
Jul 07 19:17:57.887681 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:57.887628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-b9jsw" event={"ID":"73a69ec9-8617-4360-a8c3-d903e47f4bce","Type":"ContainerStarted","Data":"7562f3f90aaa285afd527e759fcfb544f2e16d88948456844b55bcc4dc855372"}
Jul 07 19:17:59.870236 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:17:59.870144    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-659b68478d-trz2v" podUID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" containerName="console" containerID="cri-o://c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c" gracePeriod=15
Jul 07 19:18:00.639749 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.639727    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:18:00.717058 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717024    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717212 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717073    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717212 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717212 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717182    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlmq7\" (UniqueName: \"kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717337 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717212    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717337 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717337 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717303    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert\") pod \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\" (UID: \"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2\") "
Jul 07 19:18:00.717487 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717443    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:18:00.717572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717553    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-oauth-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.717659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717451    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config" (OuterVolumeSpecName: "console-config") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:18:00.717659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717631    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:18:00.717767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.717679    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:18:00.719506 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.719476    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:18:00.719506 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.719494    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:18:00.719702 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.719556    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7" (OuterVolumeSpecName: "kube-api-access-zlmq7") pod "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" (UID: "d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2"). InnerVolumeSpecName "kube-api-access-zlmq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818223    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-serving-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818250    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818264    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-trusted-ca-bundle\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818275    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlmq7\" (UniqueName: \"kubernetes.io/projected/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-kube-api-access-zlmq7\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818286    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-console-oauth-config\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.818302 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.818299    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:00.903912 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.903876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-b9jsw" event={"ID":"73a69ec9-8617-4360-a8c3-d903e47f4bce","Type":"ContainerStarted","Data":"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"}
Jul 07 19:18:00.905195 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.905164    2579 generic.go:356] "Generic (PLEG): container finished" podID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" containerID="c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c" exitCode=0
Jul 07 19:18:00.905285 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.905224    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659b68478d-trz2v"
Jul 07 19:18:00.905318 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.905224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659b68478d-trz2v" event={"ID":"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2","Type":"ContainerDied","Data":"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"}
Jul 07 19:18:00.905372 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.905361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659b68478d-trz2v" event={"ID":"d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2","Type":"ContainerDied","Data":"528badc5a6eb383b110f1b0fe05b8c0baef5da20784012f630d82cb6fc6932ca"}
Jul 07 19:18:00.905410 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.905383    2579 scope.go:117] "RemoveContainer" containerID="c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"
Jul 07 19:18:00.914155 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.914139    2579 scope.go:117] "RemoveContainer" containerID="c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"
Jul 07 19:18:00.914392 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:18:00.914374    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c\": container with ID starting with c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c not found: ID does not exist" containerID="c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"
Jul 07 19:18:00.914457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.914402    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c"} err="failed to get container status \"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c\": rpc error: code = NotFound desc = could not find container \"c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c\": container with ID starting with c59f5bc575cb78ee8a6de654c102141a46525a23f656d0532669a592166ce13c not found: ID does not exist"
Jul 07 19:18:00.920090 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.920030    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-b9jsw" podStartSLOduration=1.6524808709999999 podStartE2EDuration="4.920013102s" totalImagesPullingTime="3.267532231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:18:00.919733352 +0000 UTC m=+531.881184632" watchObservedRunningTime="2026-07-07 19:18:00.920013102 +0000 UTC m=+531.881464380"
Jul 07 19:18:00.936903 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.936881    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:18:00.942831 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:00.942803    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-659b68478d-trz2v"]
Jul 07 19:18:01.547601 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:01.547550    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" path="/var/lib/kubelet/pods/d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2/volumes"
Jul 07 19:18:02.054845 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:02.054816    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:18:02.915183 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:02.915140    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-b9jsw" podUID="73a69ec9-8617-4360-a8c3-d903e47f4bce" containerName="authorino" containerID="cri-o://a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac" gracePeriod=30
Jul 07 19:18:03.165785 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.165732    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:18:03.240205 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.240181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbw4m\" (UniqueName: \"kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m\") pod \"73a69ec9-8617-4360-a8c3-d903e47f4bce\" (UID: \"73a69ec9-8617-4360-a8c3-d903e47f4bce\") "
Jul 07 19:18:03.242384 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.242361    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m" (OuterVolumeSpecName: "kube-api-access-sbw4m") pod "73a69ec9-8617-4360-a8c3-d903e47f4bce" (UID: "73a69ec9-8617-4360-a8c3-d903e47f4bce"). InnerVolumeSpecName "kube-api-access-sbw4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:18:03.341280 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.341254    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbw4m\" (UniqueName: \"kubernetes.io/projected/73a69ec9-8617-4360-a8c3-d903e47f4bce-kube-api-access-sbw4m\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:03.920263 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.920182    2579 generic.go:356] "Generic (PLEG): container finished" podID="73a69ec9-8617-4360-a8c3-d903e47f4bce" containerID="a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac" exitCode=0
Jul 07 19:18:03.920263 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.920217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-b9jsw"
Jul 07 19:18:03.920263 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.920237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-b9jsw" event={"ID":"73a69ec9-8617-4360-a8c3-d903e47f4bce","Type":"ContainerDied","Data":"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"}
Jul 07 19:18:03.920494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.920268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-b9jsw" event={"ID":"73a69ec9-8617-4360-a8c3-d903e47f4bce","Type":"ContainerDied","Data":"7562f3f90aaa285afd527e759fcfb544f2e16d88948456844b55bcc4dc855372"}
Jul 07 19:18:03.920494 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.920285    2579 scope.go:117] "RemoveContainer" containerID="a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"
Jul 07 19:18:03.930846 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.930810    2579 scope.go:117] "RemoveContainer" containerID="a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"
Jul 07 19:18:03.931404 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:18:03.931385    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac\": container with ID starting with a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac not found: ID does not exist" containerID="a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"
Jul 07 19:18:03.931510 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.931412    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac"} err="failed to get container status \"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac\": rpc error: code = NotFound desc = could not find container \"a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac\": container with ID starting with a9a569635900404b0aad62930fb6d5ec69cc8d9e3ae48f2686b0ed9db8d87eac not found: ID does not exist"
Jul 07 19:18:03.936604 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.936571    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:18:03.941651 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:03.941631    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-b9jsw"]
Jul 07 19:18:05.548006 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:05.547975    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a69ec9-8617-4360-a8c3-d903e47f4bce" path="/var/lib/kubelet/pods/73a69ec9-8617-4360-a8c3-d903e47f4bce/volumes"
Jul 07 19:18:29.045191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045158    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:29.045806 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045680    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a69ec9-8617-4360-a8c3-d903e47f4bce" containerName="authorino"
Jul 07 19:18:29.045806 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045701    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a69ec9-8617-4360-a8c3-d903e47f4bce" containerName="authorino"
Jul 07 19:18:29.045806 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045725    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" containerName="console"
Jul 07 19:18:29.045806 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045733    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" containerName="console"
Jul 07 19:18:29.046010 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045813    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a69ec9-8617-4360-a8c3-d903e47f4bce" containerName="authorino"
Jul 07 19:18:29.046010 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.045828    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ccf495-5e67-4fe4-b060-7bbf05aeb7f2" containerName="console"
Jul 07 19:18:29.053930 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.053909    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:29.055902 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.055880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nvfkw\""
Jul 07 19:18:29.057295 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.057274    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:29.153619 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.153578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hktzr\" (UniqueName: \"kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr\") pod \"authorino-68478bdf68-22pcb\" (UID: \"55d33436-125a-4745-9860-67c0cfaffd70\") " pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:29.254420 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.254388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hktzr\" (UniqueName: \"kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr\") pod \"authorino-68478bdf68-22pcb\" (UID: \"55d33436-125a-4745-9860-67c0cfaffd70\") " pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:29.264576 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.264550    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hktzr\" (UniqueName: \"kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr\") pod \"authorino-68478bdf68-22pcb\" (UID: \"55d33436-125a-4745-9860-67c0cfaffd70\") " pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:29.364875 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.364795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:29.425965 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.425929    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:29.504517 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:29.504485    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:29.507888 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:18:29.507862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55d33436_125a_4745_9860_67c0cfaffd70.slice/crio-9666d8d18919721e1841475f9ca2abc14a377f726c566021c96438fd8d48e7c2 WatchSource:0}: Error finding container 9666d8d18919721e1841475f9ca2abc14a377f726c566021c96438fd8d48e7c2: Status 404 returned error can't find the container with id 9666d8d18919721e1841475f9ca2abc14a377f726c566021c96438fd8d48e7c2
Jul 07 19:18:30.040104 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.040069    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68478bdf68-22pcb" event={"ID":"55d33436-125a-4745-9860-67c0cfaffd70","Type":"ContainerStarted","Data":"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"}
Jul 07 19:18:30.040104 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.040087    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-68478bdf68-22pcb" podUID="55d33436-125a-4745-9860-67c0cfaffd70" containerName="authorino" containerID="cri-o://e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6" gracePeriod=30
Jul 07 19:18:30.040104 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.040114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68478bdf68-22pcb" event={"ID":"55d33436-125a-4745-9860-67c0cfaffd70","Type":"ContainerStarted","Data":"9666d8d18919721e1841475f9ca2abc14a377f726c566021c96438fd8d48e7c2"}
Jul 07 19:18:30.057783 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.057730    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68478bdf68-22pcb" podStartSLOduration=0.7327561 podStartE2EDuration="1.057711378s" totalImagesPullingTime="324.955278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:18:30.057663028 +0000 UTC m=+561.019114308" watchObservedRunningTime="2026-07-07 19:18:30.057711378 +0000 UTC m=+561.019162656"
Jul 07 19:18:30.296775 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.296748    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:30.364622 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.364568    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hktzr\" (UniqueName: \"kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr\") pod \"55d33436-125a-4745-9860-67c0cfaffd70\" (UID: \"55d33436-125a-4745-9860-67c0cfaffd70\") "
Jul 07 19:18:30.366819 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.366783    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr" (OuterVolumeSpecName: "kube-api-access-hktzr") pod "55d33436-125a-4745-9860-67c0cfaffd70" (UID: "55d33436-125a-4745-9860-67c0cfaffd70"). InnerVolumeSpecName "kube-api-access-hktzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:18:30.465847 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:30.465782    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hktzr\" (UniqueName: \"kubernetes.io/projected/55d33436-125a-4745-9860-67c0cfaffd70-kube-api-access-hktzr\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:18:31.045645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.045612    2579 generic.go:356] "Generic (PLEG): container finished" podID="55d33436-125a-4745-9860-67c0cfaffd70" containerID="e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6" exitCode=0
Jul 07 19:18:31.045645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.045648    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68478bdf68-22pcb"
Jul 07 19:18:31.045859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.045649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68478bdf68-22pcb" event={"ID":"55d33436-125a-4745-9860-67c0cfaffd70","Type":"ContainerDied","Data":"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"}
Jul 07 19:18:31.045859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.045780    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68478bdf68-22pcb" event={"ID":"55d33436-125a-4745-9860-67c0cfaffd70","Type":"ContainerDied","Data":"9666d8d18919721e1841475f9ca2abc14a377f726c566021c96438fd8d48e7c2"}
Jul 07 19:18:31.045859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.045806    2579 scope.go:117] "RemoveContainer" containerID="e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"
Jul 07 19:18:31.054167 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.054149    2579 scope.go:117] "RemoveContainer" containerID="e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"
Jul 07 19:18:31.054420 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:18:31.054398    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6\": container with ID starting with e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6 not found: ID does not exist" containerID="e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"
Jul 07 19:18:31.054473 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.054428    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6"} err="failed to get container status \"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6\": rpc error: code = NotFound desc = could not find container \"e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6\": container with ID starting with e85e82426ca590792917071185a38ef07f56caa9a3033ed98f18bd4cb805a1e6 not found: ID does not exist"
Jul 07 19:18:31.068851 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.068829    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:31.080306 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.080277    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-68478bdf68-22pcb"]
Jul 07 19:18:31.547371 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:31.547339    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d33436-125a-4745-9860-67c0cfaffd70" path="/var/lib/kubelet/pods/55d33436-125a-4745-9860-67c0cfaffd70/volumes"
Jul 07 19:18:35.840013 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.839979    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:18:35.840376 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.840300    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d33436-125a-4745-9860-67c0cfaffd70" containerName="authorino"
Jul 07 19:18:35.840376 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.840311    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d33436-125a-4745-9860-67c0cfaffd70" containerName="authorino"
Jul 07 19:18:35.840444 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.840380    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="55d33436-125a-4745-9860-67c0cfaffd70" containerName="authorino"
Jul 07 19:18:35.844177 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.844158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:35.846449 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.846425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 19:18:35.846579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.846455    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ht7q7\""
Jul 07 19:18:35.846579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.846459    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 19:18:35.853203 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.853172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:18:35.912223 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.912199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:35.912360 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.912247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:35.912360 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:35.912280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjrh7\" (UniqueName: \"kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.013127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.013100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.013257 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.013138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.013257 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.013163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjrh7\" (UniqueName: \"kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.013835 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.013810    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.015700 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.015682    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.023350 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.023331    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjrh7\" (UniqueName: \"kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7\") pod \"maas-controller-6847f4bd59-lz4pt\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") " pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.155530 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.155465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:36.313900 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:36.313873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:18:36.315944 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:18:36.315915    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5665f6e4_f32c_490e_b996_5b6093a4d417.slice/crio-621aaede82140cb37bbaa46ed4c88ae5d08c6f0ff5b895967350f6e28e8c62f1 WatchSource:0}: Error finding container 621aaede82140cb37bbaa46ed4c88ae5d08c6f0ff5b895967350f6e28e8c62f1: Status 404 returned error can't find the container with id 621aaede82140cb37bbaa46ed4c88ae5d08c6f0ff5b895967350f6e28e8c62f1
Jul 07 19:18:37.072899 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:37.072851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" event={"ID":"5665f6e4-f32c-490e-b996-5b6093a4d417","Type":"ContainerStarted","Data":"621aaede82140cb37bbaa46ed4c88ae5d08c6f0ff5b895967350f6e28e8c62f1"}
Jul 07 19:18:39.087199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:39.087157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" event={"ID":"5665f6e4-f32c-490e-b996-5b6093a4d417","Type":"ContainerStarted","Data":"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"}
Jul 07 19:18:39.087616 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:39.087327    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:18:39.110550 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:39.110507    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" podStartSLOduration=1.817852728 podStartE2EDuration="4.110493322s" totalImagesPullingTime="2.292640594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:18:39.109503014 +0000 UTC m=+570.070954288" watchObservedRunningTime="2026-07-07 19:18:39.110493322 +0000 UTC m=+570.071944599"
Jul 07 19:18:50.098972 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:18:50.098943    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:19:11.709576 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.709546    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-txnbg"]
Jul 07 19:19:11.716042 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.716024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.718242 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.718225    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 19:19:11.718345 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.718228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-64h7f\""
Jul 07 19:19:11.724893 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.724875    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-txnbg"]
Jul 07 19:19:11.813461 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.813431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-plugins-config-volume\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.813461 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.813464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt7v5\" (UniqueName: \"kubernetes.io/projected/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-kube-api-access-lt7v5\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.914088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.914056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-plugins-config-volume\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.914088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.914091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt7v5\" (UniqueName: \"kubernetes.io/projected/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-kube-api-access-lt7v5\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.914802 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.914785    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-plugins-config-volume\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:11.928466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:11.928433    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt7v5\" (UniqueName: \"kubernetes.io/projected/20f6a441-f584-4a2e-b1da-6b022d8c1a5d-kube-api-access-lt7v5\") pod \"payload-processing-58f86c7798-txnbg\" (UID: \"20f6a441-f584-4a2e-b1da-6b022d8c1a5d\") " pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:12.027436 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:12.027406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:12.161736 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:12.161696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-txnbg"]
Jul 07 19:19:12.164049 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:19:12.164023    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f6a441_f584_4a2e_b1da_6b022d8c1a5d.slice/crio-e7109cd9813dee8810b1439336f48943c2eeb0e93825c7cf47c6b16bada3cdc1 WatchSource:0}: Error finding container e7109cd9813dee8810b1439336f48943c2eeb0e93825c7cf47c6b16bada3cdc1: Status 404 returned error can't find the container with id e7109cd9813dee8810b1439336f48943c2eeb0e93825c7cf47c6b16bada3cdc1
Jul 07 19:19:12.231417 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:12.231388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-txnbg" event={"ID":"20f6a441-f584-4a2e-b1da-6b022d8c1a5d","Type":"ContainerStarted","Data":"e7109cd9813dee8810b1439336f48943c2eeb0e93825c7cf47c6b16bada3cdc1"}
Jul 07 19:19:14.242071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:14.242034    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-txnbg" event={"ID":"20f6a441-f584-4a2e-b1da-6b022d8c1a5d","Type":"ContainerStarted","Data":"812fcfbd88da4f74b1e9a82dfb2d76e25fb503c84604e49e8412e104bf17d71c"}
Jul 07 19:19:14.242461 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:14.242165    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:14.269405 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:14.269355    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-txnbg" podStartSLOduration=1.718161627 podStartE2EDuration="3.269340378s" totalImagesPullingTime="1.551178751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:19:14.266582788 +0000 UTC m=+605.228034066" watchObservedRunningTime="2026-07-07 19:19:14.269340378 +0000 UTC m=+605.230791654"
Jul 07 19:19:25.248443 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:25.248410    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-txnbg"
Jul 07 19:19:50.803901 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.803864    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"]
Jul 07 19:19:50.806050 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.806033    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.808079 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.808059    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 19:19:50.808526 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.808506    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 19:19:50.808769 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.808752    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 19:19:50.808833 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.808759    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-l9d5q\""
Jul 07 19:19:50.816952 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.816931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"]
Jul 07 19:19:50.841395 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.841512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.841512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.841512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.841675 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b484q\" (UniqueName: \"kubernetes.io/projected/a4910f70-d42d-4e6e-a61b-038c58d9f77c-kube-api-access-b484q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.841675 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.841652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942457 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b484q\" (UniqueName: \"kubernetes.io/projected/a4910f70-d42d-4e6e-a61b-038c58d9f77c-kube-api-access-b484q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942640 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942640 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942640 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942640 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942640 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942918 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942899    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.942978 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.942915    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.943105 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.943084    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.945106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.945078    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4910f70-d42d-4e6e-a61b-038c58d9f77c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.945106 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.945088    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4910f70-d42d-4e6e-a61b-038c58d9f77c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:50.951767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:50.951735    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b484q\" (UniqueName: \"kubernetes.io/projected/a4910f70-d42d-4e6e-a61b-038c58d9f77c-kube-api-access-b484q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2bmfr\" (UID: \"a4910f70-d42d-4e6e-a61b-038c58d9f77c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:51.117719 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:51.117634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:19:51.251916 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:51.251893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"]
Jul 07 19:19:51.254222 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:19:51.254194    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4910f70_d42d_4e6e_a61b_038c58d9f77c.slice/crio-41e367a307c7c676163f25119dffbd5a0aee0ee2354ab1d69469e4a06d594b16 WatchSource:0}: Error finding container 41e367a307c7c676163f25119dffbd5a0aee0ee2354ab1d69469e4a06d594b16: Status 404 returned error can't find the container with id 41e367a307c7c676163f25119dffbd5a0aee0ee2354ab1d69469e4a06d594b16
Jul 07 19:19:51.255508 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:51.255492    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 19:19:51.409581 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:51.409501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr" event={"ID":"a4910f70-d42d-4e6e-a61b-038c58d9f77c","Type":"ContainerStarted","Data":"41e367a307c7c676163f25119dffbd5a0aee0ee2354ab1d69469e4a06d594b16"}
Jul 07 19:19:52.416008 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:52.415975    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr" event={"ID":"a4910f70-d42d-4e6e-a61b-038c58d9f77c","Type":"ContainerStarted","Data":"e7a7b0eb796c2b329468c1f38253bcdf1a526dd938d34c02409490a06ca50880"}
Jul 07 19:19:52.436936 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:19:52.436887    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr" podStartSLOduration=1.43955342 podStartE2EDuration="2.436871375s" totalImagesPullingTime="997.317955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:19:52.435495633 +0000 UTC m=+643.396946909" watchObservedRunningTime="2026-07-07 19:19:52.436871375 +0000 UTC m=+643.398322652"
Jul 07 19:20:01.118333 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:01.118296    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:20:01.118751 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:01.118368    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:20:01.131026 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:01.130999    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:20:01.467473 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:01.467447    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2bmfr"
Jul 07 19:20:14.607645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.607610    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"]
Jul 07 19:20:14.611200 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.611183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.613577 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.613552    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 07 19:20:14.620552 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.620525    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"]
Jul 07 19:20:14.752999 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.752963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.753186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.753010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.753186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.753087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.753186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.753127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.753186 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.753167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds4vl\" (UniqueName: \"kubernetes.io/projected/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-kube-api-access-ds4vl\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.753369 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.753210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854030 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.853991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854030 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds4vl\" (UniqueName: \"kubernetes.io/projected/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-kube-api-access-ds4vl\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854279 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854513 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854495    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854558 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854529    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.854674 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.854645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.856512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.856490    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.856665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.856649    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.865262 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.865183    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds4vl\" (UniqueName: \"kubernetes.io/projected/d02ead4d-41fb-496e-a5bd-af8c183a5ceb-kube-api-access-ds4vl\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xzf4f\" (UID: \"d02ead4d-41fb-496e-a5bd-af8c183a5ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:14.924597 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:14.924560    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:15.061544 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:15.061516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"]
Jul 07 19:20:15.063513 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:20:15.063477    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02ead4d_41fb_496e_a5bd_af8c183a5ceb.slice/crio-a0b4b37c6096af7534eaa79631d2f8252725ab75ae65acc28ad040b8983dcd0e WatchSource:0}: Error finding container a0b4b37c6096af7534eaa79631d2f8252725ab75ae65acc28ad040b8983dcd0e: Status 404 returned error can't find the container with id a0b4b37c6096af7534eaa79631d2f8252725ab75ae65acc28ad040b8983dcd0e
Jul 07 19:20:15.517521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:15.517434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f" event={"ID":"d02ead4d-41fb-496e-a5bd-af8c183a5ceb","Type":"ContainerStarted","Data":"2bc0a96f9be37212393e73ca433c7cb042b597e14c573ab09eaa2d4bd076b6e7"}
Jul 07 19:20:15.517521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:15.517489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f" event={"ID":"d02ead4d-41fb-496e-a5bd-af8c183a5ceb","Type":"ContainerStarted","Data":"a0b4b37c6096af7534eaa79631d2f8252725ab75ae65acc28ad040b8983dcd0e"}
Jul 07 19:20:15.536897 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:15.536843    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f" podStartSLOduration=1.347218378 podStartE2EDuration="1.536823105s" totalImagesPullingTime="189.604727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:20:15.536502987 +0000 UTC m=+666.497954276" watchObservedRunningTime="2026-07-07 19:20:15.536823105 +0000 UTC m=+666.498274383"
Jul 07 19:20:24.925166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:24.925127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:24.925698 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:24.925185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:24.937629 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:24.937603    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:25.574390 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:25.574357    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xzf4f"
Jul 07 19:20:26.005259 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.005221    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"]
Jul 07 19:20:26.011380 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.011358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.013249 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.013228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 19:20:26.024676 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.024651    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"]
Jul 07 19:20:26.154713 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.154896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2gj\" (UniqueName: \"kubernetes.io/projected/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-kube-api-access-hv2gj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.154896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.154896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154841    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.154896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.155105 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.154933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2gj\" (UniqueName: \"kubernetes.io/projected/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-kube-api-access-hv2gj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256149 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256419 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256419 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256419 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256566 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256517    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256566 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256556    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.256701 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.256680    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.258537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.258511    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.258865 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.258846    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.265032 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.265007    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2gj\" (UniqueName: \"kubernetes.io/projected/e1be8192-9e2b-41c4-a7a2-5c434ef247d8-kube-api-access-hv2gj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2\" (UID: \"e1be8192-9e2b-41c4-a7a2-5c434ef247d8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.320908 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.320880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:26.459381 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.459352    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"]
Jul 07 19:20:26.461143 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:20:26.461115    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1be8192_9e2b_41c4_a7a2_5c434ef247d8.slice/crio-2225a67dc0b2c88258a206b0f9d0d34e22424a88dbe3b1d3cb4cc02acff3c24e WatchSource:0}: Error finding container 2225a67dc0b2c88258a206b0f9d0d34e22424a88dbe3b1d3cb4cc02acff3c24e: Status 404 returned error can't find the container with id 2225a67dc0b2c88258a206b0f9d0d34e22424a88dbe3b1d3cb4cc02acff3c24e
Jul 07 19:20:26.569371 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:26.569284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2" event={"ID":"e1be8192-9e2b-41c4-a7a2-5c434ef247d8","Type":"ContainerStarted","Data":"2225a67dc0b2c88258a206b0f9d0d34e22424a88dbe3b1d3cb4cc02acff3c24e"}
Jul 07 19:20:27.575726 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:27.575688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2" event={"ID":"e1be8192-9e2b-41c4-a7a2-5c434ef247d8","Type":"ContainerStarted","Data":"ac52715ea652f2921717f67982e3efa266e9448eff9c76c51f366e7bc51a95f9"}
Jul 07 19:20:27.596797 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:27.596739    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2" podStartSLOduration=2.412239156 podStartE2EDuration="2.596723651s" totalImagesPullingTime="184.484495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:20:27.594559123 +0000 UTC m=+678.556010403" watchObservedRunningTime="2026-07-07 19:20:27.596723651 +0000 UTC m=+678.558174927"
Jul 07 19:20:36.321702 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:36.321666    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:36.322188 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:36.321951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:36.334336 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:36.334310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:20:36.627502 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:20:36.627430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2"
Jul 07 19:23:18.130006 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.129976    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:23:18.130539 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.130172    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" podUID="5665f6e4-f32c-490e-b996-5b6093a4d417" containerName="manager" containerID="cri-o://6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed" gracePeriod=30
Jul 07 19:23:18.378213 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.378190    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:23:18.378990 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.378969    2579 generic.go:356] "Generic (PLEG): container finished" podID="5665f6e4-f32c-490e-b996-5b6093a4d417" containerID="6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed" exitCode=0
Jul 07 19:23:18.379084 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.379036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" event={"ID":"5665f6e4-f32c-490e-b996-5b6093a4d417","Type":"ContainerDied","Data":"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"}
Jul 07 19:23:18.379124 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.379083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847f4bd59-lz4pt" event={"ID":"5665f6e4-f32c-490e-b996-5b6093a4d417","Type":"ContainerDied","Data":"621aaede82140cb37bbaa46ed4c88ae5d08c6f0ff5b895967350f6e28e8c62f1"}
Jul 07 19:23:18.379124 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.379106    2579 scope.go:117] "RemoveContainer" containerID="6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"
Jul 07 19:23:18.387444 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.387428    2579 scope.go:117] "RemoveContainer" containerID="6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"
Jul 07 19:23:18.387714 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:23:18.387697    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed\": container with ID starting with 6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed not found: ID does not exist" containerID="6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"
Jul 07 19:23:18.387763 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.387722    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed"} err="failed to get container status \"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed\": rpc error: code = NotFound desc = could not find container \"6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed\": container with ID starting with 6723e31605d83e27dee627ca4d2e3f14580b8acd97f6478149d1dae4c46238ed not found: ID does not exist"
Jul 07 19:23:18.500898 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.500869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca\") pod \"5665f6e4-f32c-490e-b996-5b6093a4d417\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") "
Jul 07 19:23:18.501056 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.500961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjrh7\" (UniqueName: \"kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7\") pod \"5665f6e4-f32c-490e-b996-5b6093a4d417\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") "
Jul 07 19:23:18.501056 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.501041    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert\") pod \"5665f6e4-f32c-490e-b996-5b6093a4d417\" (UID: \"5665f6e4-f32c-490e-b996-5b6093a4d417\") "
Jul 07 19:23:18.501264 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.501235    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca" (OuterVolumeSpecName: "service-ca") pod "5665f6e4-f32c-490e-b996-5b6093a4d417" (UID: "5665f6e4-f32c-490e-b996-5b6093a4d417"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 19:23:18.503240 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.503209    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5665f6e4-f32c-490e-b996-5b6093a4d417" (UID: "5665f6e4-f32c-490e-b996-5b6093a4d417"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:23:18.503383 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.503362    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7" (OuterVolumeSpecName: "kube-api-access-vjrh7") pod "5665f6e4-f32c-490e-b996-5b6093a4d417" (UID: "5665f6e4-f32c-490e-b996-5b6093a4d417"). InnerVolumeSpecName "kube-api-access-vjrh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:23:18.602468 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.602427    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjrh7\" (UniqueName: \"kubernetes.io/projected/5665f6e4-f32c-490e-b996-5b6093a4d417-kube-api-access-vjrh7\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:23:18.602468 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.602465    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5665f6e4-f32c-490e-b996-5b6093a4d417-webhook-cert\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:23:18.602468 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:18.602475    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5665f6e4-f32c-490e-b996-5b6093a4d417-service-ca\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:23:19.384918 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:19.384890    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847f4bd59-lz4pt"
Jul 07 19:23:19.408766 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:19.408738    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:23:19.414381 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:19.414359    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6847f4bd59-lz4pt"]
Jul 07 19:23:19.548708 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:23:19.548678    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5665f6e4-f32c-490e-b996-5b6093a4d417" path="/var/lib/kubelet/pods/5665f6e4-f32c-490e-b996-5b6093a4d417/volumes"
Jul 07 19:28:06.426603 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:06.426550    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:28:06.427173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:06.426859    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" podUID="a2a89d86-69df-45c5-8527-4c02526cee8b" containerName="manager" containerID="cri-o://1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0" gracePeriod=10
Jul 07 19:28:08.684410 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.684380    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:28:08.721751 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.721714    2579 generic.go:356] "Generic (PLEG): container finished" podID="a2a89d86-69df-45c5-8527-4c02526cee8b" containerID="1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0" exitCode=0
Jul 07 19:28:08.721911 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.721771    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"
Jul 07 19:28:08.721911 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.721763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" event={"ID":"a2a89d86-69df-45c5-8527-4c02526cee8b","Type":"ContainerDied","Data":"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"}
Jul 07 19:28:08.722034 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.721908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll" event={"ID":"a2a89d86-69df-45c5-8527-4c02526cee8b","Type":"ContainerDied","Data":"63cef05f83af3b9c2b268122bfe954ede4b17111fafb158f7bd2257f1e5558a1"}
Jul 07 19:28:08.722034 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.721946    2579 scope.go:117] "RemoveContainer" containerID="1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"
Jul 07 19:28:08.731739 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.731722    2579 scope.go:117] "RemoveContainer" containerID="1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"
Jul 07 19:28:08.732010 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:28:08.731992    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0\": container with ID starting with 1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0 not found: ID does not exist" containerID="1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"
Jul 07 19:28:08.732063 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.732017    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0"} err="failed to get container status \"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0\": rpc error: code = NotFound desc = could not find container \"1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0\": container with ID starting with 1366aec54b29633687294b8540ebbcbfc69ec2317421aad5687c50a9b01371e0 not found: ID does not exist"
Jul 07 19:28:08.736293 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.736263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhsfz\" (UniqueName: \"kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz\") pod \"a2a89d86-69df-45c5-8527-4c02526cee8b\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") "
Jul 07 19:28:08.736409 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.736332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume\") pod \"a2a89d86-69df-45c5-8527-4c02526cee8b\" (UID: \"a2a89d86-69df-45c5-8527-4c02526cee8b\") "
Jul 07 19:28:08.736705 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.736684    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a2a89d86-69df-45c5-8527-4c02526cee8b" (UID: "a2a89d86-69df-45c5-8527-4c02526cee8b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:28:08.738452 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.738427    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz" (OuterVolumeSpecName: "kube-api-access-rhsfz") pod "a2a89d86-69df-45c5-8527-4c02526cee8b" (UID: "a2a89d86-69df-45c5-8527-4c02526cee8b"). InnerVolumeSpecName "kube-api-access-rhsfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:28:08.837418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.837335    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhsfz\" (UniqueName: \"kubernetes.io/projected/a2a89d86-69df-45c5-8527-4c02526cee8b-kube-api-access-rhsfz\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:28:08.837418 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:08.837361    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2a89d86-69df-45c5-8527-4c02526cee8b-extensions-socket-volume\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:28:09.045880 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:09.045850    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:28:09.050293 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:09.050270    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxsll"]
Jul 07 19:28:09.548117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:28:09.548086    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2a89d86-69df-45c5-8527-4c02526cee8b" path="/var/lib/kubelet/pods/a2a89d86-69df-45c5-8527-4c02526cee8b/volumes"
Jul 07 19:29:07.574496 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574419    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"]
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a89d86-69df-45c5-8527-4c02526cee8b" containerName="manager"
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574787    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a89d86-69df-45c5-8527-4c02526cee8b" containerName="manager"
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574797    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5665f6e4-f32c-490e-b996-5b6093a4d417" containerName="manager"
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574803    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5665f6e4-f32c-490e-b996-5b6093a4d417" containerName="manager"
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574876    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5665f6e4-f32c-490e-b996-5b6093a4d417" containerName="manager"
Jul 07 19:29:07.575029 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.574885    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a89d86-69df-45c5-8527-4c02526cee8b" containerName="manager"
Jul 07 19:29:07.577880 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.577861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.580692 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.580674    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q5b5p\""
Jul 07 19:29:07.614681 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.614657    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"]
Jul 07 19:29:07.670683 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.670658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.670812 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.670696    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6t9g\" (UniqueName: \"kubernetes.io/projected/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-kube-api-access-n6t9g\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.771360 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.771324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6t9g\" (UniqueName: \"kubernetes.io/projected/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-kube-api-access-n6t9g\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.771500 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.771428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.771814 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.771795    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.782221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.782195    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6t9g\" (UniqueName: \"kubernetes.io/projected/d2d1bd53-29ba-4ac0-86e0-afd1b1e04766-kube-api-access-n6t9g\") pod \"kuadrant-operator-controller-manager-86878c769f-mttmz\" (UID: \"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:07.887735 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:07.887646    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:08.032865 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:08.032838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"]
Jul 07 19:29:08.035092 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:29:08.035065    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d1bd53_29ba_4ac0_86e0_afd1b1e04766.slice/crio-46d044d8236c0d9f5d3b200abc6d0e9fa6949da06cba1ffef0bdd535c86e8921 WatchSource:0}: Error finding container 46d044d8236c0d9f5d3b200abc6d0e9fa6949da06cba1ffef0bdd535c86e8921: Status 404 returned error can't find the container with id 46d044d8236c0d9f5d3b200abc6d0e9fa6949da06cba1ffef0bdd535c86e8921
Jul 07 19:29:08.998119 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:08.998083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz" event={"ID":"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766","Type":"ContainerStarted","Data":"61b5f9c83b8921091e7f5017891a714ad1ff75292165470c0f4c7c45a35e6d95"}
Jul 07 19:29:08.998565 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:08.998132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz" event={"ID":"d2d1bd53-29ba-4ac0-86e0-afd1b1e04766","Type":"ContainerStarted","Data":"46d044d8236c0d9f5d3b200abc6d0e9fa6949da06cba1ffef0bdd535c86e8921"}
Jul 07 19:29:08.998565 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:08.998177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:29:09.022037 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:09.021994    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz" podStartSLOduration=2.021981133 podStartE2EDuration="2.021981133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:09.019162192 +0000 UTC m=+1199.980613481" watchObservedRunningTime="2026-07-07 19:29:09.021981133 +0000 UTC m=+1199.983432410"
Jul 07 19:29:20.005760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:29:20.005707    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mttmz"
Jul 07 19:39:20.608308 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.608272    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:39:20.611825 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.611807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.613480 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.613460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6fe61d50\"/\"default-dockercfg-bdgqr\""
Jul 07 19:39:20.613701 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.613681    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6fe61d50\"/\"kube-root-ca.crt\""
Jul 07 19:39:20.614074 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.614059    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6fe61d50\"/\"openshift-service-ca.crt\""
Jul 07 19:39:20.614141 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.614122    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6fe61d50\"/\"e2e-lifecycle-model-6fe61d50-kserve-self-signed-certs\""
Jul 07 19:39:20.620804 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.620784    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:39:20.661987 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.661966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.662117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.661996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.662117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.662024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v69v\" (UniqueName: \"kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.662199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.662132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.662240 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.662194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.662240 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.662223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.762953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.762924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763123 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.762967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763123 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763123 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763123 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v69v\" (UniqueName: \"kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763306 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763427 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763406    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763440    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.763521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.763466    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.765532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.765507    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.765711 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.765693    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.771516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.771494    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v69v\" (UniqueName: \"kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v\") pod \"e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") " pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:20.923380 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:20.923300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:21.054792 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:21.054673    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:39:21.057449 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:39:21.057420    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2f88f2a_5a17_4604_92c9_a2ce5f20534a.slice/crio-5fd3cf2bec5d47488629df792f06086bbd056f00f8b17a2837de0377ad5e4895 WatchSource:0}: Error finding container 5fd3cf2bec5d47488629df792f06086bbd056f00f8b17a2837de0377ad5e4895: Status 404 returned error can't find the container with id 5fd3cf2bec5d47488629df792f06086bbd056f00f8b17a2837de0377ad5e4895
Jul 07 19:39:21.821087 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:21.821057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" event={"ID":"e2f88f2a-5a17-4604-92c9-a2ce5f20534a","Type":"ContainerStarted","Data":"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"}
Jul 07 19:39:21.821445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:21.821094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" event={"ID":"e2f88f2a-5a17-4604-92c9-a2ce5f20534a","Type":"ContainerStarted","Data":"5fd3cf2bec5d47488629df792f06086bbd056f00f8b17a2837de0377ad5e4895"}
Jul 07 19:39:21.837928 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:21.837878    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" podStartSLOduration=1.8378592409999999 podStartE2EDuration="1.837859241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:39:21.837225575 +0000 UTC m=+1812.798676852" watchObservedRunningTime="2026-07-07 19:39:21.837859241 +0000 UTC m=+1812.799310521"
Jul 07 19:39:30.923720 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:30.923674    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:30.924184 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:30.923748    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:30.936070 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:30.936051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:31.878371 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:31.878343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:39:51.625620 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:39:51.625572    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:40:06.647627 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.647566    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" podUID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" containerName="main" containerID="cri-o://8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e" gracePeriod=45
Jul 07 19:40:06.899903 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.899848    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:40:06.978579 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.978773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978609    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.978773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978630    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.978773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.978773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978682    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v69v\" (UniqueName: \"kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.978773 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978702    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs\") pod \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\" (UID: \"e2f88f2a-5a17-4604-92c9-a2ce5f20534a\") "
Jul 07 19:40:06.979039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978827    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home" (OuterVolumeSpecName: "home") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:40:06.979039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978911    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:40:06.979039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.978926    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache" (OuterVolumeSpecName: "model-cache") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:40:06.979154 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.979047    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:06.979154 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.979065    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:06.979154 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.979081    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:06.980964 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.980938    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:40:06.981047 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.980975    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v" (OuterVolumeSpecName: "kube-api-access-8v69v") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "kube-api-access-8v69v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:40:06.981098 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:06.981055    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm" (OuterVolumeSpecName: "dshm") pod "e2f88f2a-5a17-4604-92c9-a2ce5f20534a" (UID: "e2f88f2a-5a17-4604-92c9-a2ce5f20534a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:40:07.024069 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.024035    2579 generic.go:356] "Generic (PLEG): container finished" podID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" containerID="8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e" exitCode=0
Jul 07 19:40:07.024191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.024081    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"
Jul 07 19:40:07.024191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.024094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" event={"ID":"e2f88f2a-5a17-4604-92c9-a2ce5f20534a","Type":"ContainerDied","Data":"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"}
Jul 07 19:40:07.024191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.024133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l" event={"ID":"e2f88f2a-5a17-4604-92c9-a2ce5f20534a","Type":"ContainerDied","Data":"5fd3cf2bec5d47488629df792f06086bbd056f00f8b17a2837de0377ad5e4895"}
Jul 07 19:40:07.024191 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.024156    2579 scope.go:117] "RemoveContainer" containerID="8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"
Jul 07 19:40:07.033438 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.033422    2579 scope.go:117] "RemoveContainer" containerID="8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"
Jul 07 19:40:07.033719 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:40:07.033697    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e\": container with ID starting with 8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e not found: ID does not exist" containerID="8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"
Jul 07 19:40:07.033811 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.033726    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e"} err="failed to get container status \"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e\": rpc error: code = NotFound desc = could not find container \"8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e\": container with ID starting with 8305483e2e7972a1104b4fa7b2f476a8b22f3e1618af05cb8609cca38162bf9e not found: ID does not exist"
Jul 07 19:40:07.046536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.046514    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:40:07.052098 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.052078    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6fe61d50/e2e-lifecycle-model-6fe61d50-kserve-5d747685cf-jsx9l"]
Jul 07 19:40:07.079935 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.079910    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:07.079935 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.079935    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v69v\" (UniqueName: \"kubernetes.io/projected/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-kube-api-access-8v69v\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:07.080086 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.079946    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2f88f2a-5a17-4604-92c9-a2ce5f20534a-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:40:07.547778 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:40:07.547727    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" path="/var/lib/kubelet/pods/e2f88f2a-5a17-4604-92c9-a2ce5f20534a/volumes"
Jul 07 19:42:09.829034 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.828950    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:42:09.829574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.829522    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" containerName="main"
Jul 07 19:42:09.829574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.829543    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" containerName="main"
Jul 07 19:42:09.829718 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.829674    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f88f2a-5a17-4604-92c9-a2ce5f20534a" containerName="main"
Jul 07 19:42:09.832789 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.832767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.834924 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.834906    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1aa2d9\"/\"kube-root-ca.crt\""
Jul 07 19:42:09.835304 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.835274    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1aa2d9\"/\"openshift-service-ca.crt\""
Jul 07 19:42:09.835304 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.835294    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1aa2d9\"/\"default-dockercfg-nvsxz\""
Jul 07 19:42:09.835624 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.835609    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1aa2d9\"/\"test-model-1aa2d9-kserve-self-signed-certs\""
Jul 07 19:42:09.850091 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.850068    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:42:09.881387 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.881549 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.881549 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2w9\" (UniqueName: \"kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.881549 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.881549 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.881846 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.881559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983042 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983203 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983255 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2w9\" (UniqueName: \"kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983299 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983250    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983299 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983376 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983431 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983411    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983471 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983427    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.983677 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.983659    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.985914 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.985894    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:09.986093 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:09.986076    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:10.003020 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.002996    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2w9\" (UniqueName: \"kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9\") pod \"test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") " pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:10.141997 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.141912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:10.293725 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.293700    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:42:10.295406 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:42:10.295381    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b49ed6b_4b1b_4576_b4be_3b4499f8b57c.slice/crio-c0927b4e86cc5859bf3d1b6a9a43ebb85c7ac8cdda7ab7263b35aeca2d4ffa3b WatchSource:0}: Error finding container c0927b4e86cc5859bf3d1b6a9a43ebb85c7ac8cdda7ab7263b35aeca2d4ffa3b: Status 404 returned error can't find the container with id c0927b4e86cc5859bf3d1b6a9a43ebb85c7ac8cdda7ab7263b35aeca2d4ffa3b
Jul 07 19:42:10.597827 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.597791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" event={"ID":"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c","Type":"ContainerStarted","Data":"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"}
Jul 07 19:42:10.597827 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.597834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" event={"ID":"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c","Type":"ContainerStarted","Data":"c0927b4e86cc5859bf3d1b6a9a43ebb85c7ac8cdda7ab7263b35aeca2d4ffa3b"}
Jul 07 19:42:10.634934 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:10.634885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" podStartSLOduration=1.63486857 podStartE2EDuration="1.63486857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:42:10.634430489 +0000 UTC m=+1981.595881765" watchObservedRunningTime="2026-07-07 19:42:10.63486857 +0000 UTC m=+1981.596319847"
Jul 07 19:42:20.142236 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:20.142197    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:20.142729 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:20.142260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:20.155683 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:20.155646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:20.663019 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:20.662986    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:42:26.547462 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.547418    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:42:26.551088 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.551071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.553162 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.553138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c47ca8\"/\"kube-root-ca.crt\""
Jul 07 19:42:26.553634 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.553614    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c47ca8\"/\"test-model-c47ca8-kserve-self-signed-certs\""
Jul 07 19:42:26.553727 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.553631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c47ca8\"/\"openshift-service-ca.crt\""
Jul 07 19:42:26.553727 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.553616    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c47ca8\"/\"default-dockercfg-7qc2z\""
Jul 07 19:42:26.565935 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.565914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:42:26.628833 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.628800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.628992 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.628836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.629093 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.629059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.629294 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.629257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.629382 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.629335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfz5d\" (UniqueName: \"kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.629568 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.629532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730381 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfz5d\" (UniqueName: \"kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730487    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730812 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730790    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730939 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730914    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.730939 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.730933    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.732726 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.732703    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.733168 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.733154    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.742888 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.742865    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfz5d\" (UniqueName: \"kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d\") pod \"test-model-c47ca8-kserve-5688579df-47nn6\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") " pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:26.861402 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:26.861314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:27.008526 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:27.008497    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:42:27.010617 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:42:27.010574    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77361d36_615d_4e15_9927_c1def34d26df.slice/crio-1af7b548af7e9f6adbae33562b228227277301e35ae9af6de80388ee20bc2296 WatchSource:0}: Error finding container 1af7b548af7e9f6adbae33562b228227277301e35ae9af6de80388ee20bc2296: Status 404 returned error can't find the container with id 1af7b548af7e9f6adbae33562b228227277301e35ae9af6de80388ee20bc2296
Jul 07 19:42:27.685386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:27.685352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" event={"ID":"77361d36-615d-4e15-9927-c1def34d26df","Type":"ContainerStarted","Data":"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"}
Jul 07 19:42:27.685386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:27.685392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" event={"ID":"77361d36-615d-4e15-9927-c1def34d26df","Type":"ContainerStarted","Data":"1af7b548af7e9f6adbae33562b228227277301e35ae9af6de80388ee20bc2296"}
Jul 07 19:42:27.710229 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:27.710145    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" podStartSLOduration=1.7101248949999999 podStartE2EDuration="1.710124895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:42:27.709104821 +0000 UTC m=+1998.670556098" watchObservedRunningTime="2026-07-07 19:42:27.710124895 +0000 UTC m=+1998.671576173"
Jul 07 19:42:36.862117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:36.862077    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:36.862117 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:36.862125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:36.874948 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:36.874924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:42:37.742952 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:42:37.742921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:43:02.824664 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.824630    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:02.828224 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.828207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.830684 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.830661    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-a48272\"/\"openshift-service-ca.crt\""
Jul 07 19:43:02.830895 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.830680    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-a48272\"/\"test-model-a48272-kserve-self-signed-certs\""
Jul 07 19:43:02.830895 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.830722    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-a48272\"/\"default-dockercfg-8hg48\""
Jul 07 19:43:02.830895 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.830741    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-a48272\"/\"kube-root-ca.crt\""
Jul 07 19:43:02.846532 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.846671 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.846671 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846562    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.846671 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.846831 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846724    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.846831 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.846815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p97fc\" (UniqueName: \"kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.851493 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.850799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:02.947621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.947822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p97fc\" (UniqueName: \"kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.947822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.947822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.947822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.947822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.947743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.948141 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.948116    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.948215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.948196    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.948272 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.948224    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.949994 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.949972    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.950358 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.950343    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:02.956866 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:02.956847    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p97fc\" (UniqueName: \"kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc\") pod \"test-model-a48272-kserve-68d64b7c85-6x92v\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") " pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:03.137704 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:03.137616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:03.277401 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:03.277371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:03.279607 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:43:03.279563    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1af783e5_7577_4961_85ae_6170e93ce4ac.slice/crio-9c55b13ea2c3469637fa497a4e39c2cb2f39c68e2e4ffcd9d91b4651f772f67b WatchSource:0}: Error finding container 9c55b13ea2c3469637fa497a4e39c2cb2f39c68e2e4ffcd9d91b4651f772f67b: Status 404 returned error can't find the container with id 9c55b13ea2c3469637fa497a4e39c2cb2f39c68e2e4ffcd9d91b4651f772f67b
Jul 07 19:43:03.854916 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:03.854882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" event={"ID":"1af783e5-7577-4961-85ae-6170e93ce4ac","Type":"ContainerStarted","Data":"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"}
Jul 07 19:43:03.854916 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:03.854923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" event={"ID":"1af783e5-7577-4961-85ae-6170e93ce4ac","Type":"ContainerStarted","Data":"9c55b13ea2c3469637fa497a4e39c2cb2f39c68e2e4ffcd9d91b4651f772f67b"}
Jul 07 19:43:03.877530 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:03.877480    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" podStartSLOduration=1.877462856 podStartE2EDuration="1.877462856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:43:03.875582223 +0000 UTC m=+2034.837033497" watchObservedRunningTime="2026-07-07 19:43:03.877462856 +0000 UTC m=+2034.838914133"
Jul 07 19:43:13.138037 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:13.138006    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:13.138487 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:13.138055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:13.154759 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:13.154731    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:13.911683 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:13.911655    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:22.698553 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:22.698516    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:37.719210 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:37.719124    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" podUID="1af783e5-7577-4961-85ae-6170e93ce4ac" containerName="main" containerID="cri-o://5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0" gracePeriod=45
Jul 07 19:43:37.968016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:37.967990    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:38.019791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.019715    2579 generic.go:356] "Generic (PLEG): container finished" podID="1af783e5-7577-4961-85ae-6170e93ce4ac" containerID="5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0" exitCode=0
Jul 07 19:43:38.019791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.019758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" event={"ID":"1af783e5-7577-4961-85ae-6170e93ce4ac","Type":"ContainerDied","Data":"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"}
Jul 07 19:43:38.019791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.019770    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"
Jul 07 19:43:38.019791 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.019784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v" event={"ID":"1af783e5-7577-4961-85ae-6170e93ce4ac","Type":"ContainerDied","Data":"9c55b13ea2c3469637fa497a4e39c2cb2f39c68e2e4ffcd9d91b4651f772f67b"}
Jul 07 19:43:38.020070 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.019799    2579 scope.go:117] "RemoveContainer" containerID="5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"
Jul 07 19:43:38.030080 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.030057    2579 scope.go:117] "RemoveContainer" containerID="5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"
Jul 07 19:43:38.030361 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:43:38.030343    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0\": container with ID starting with 5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0 not found: ID does not exist" containerID="5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"
Jul 07 19:43:38.030409 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.030369    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0"} err="failed to get container status \"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0\": rpc error: code = NotFound desc = could not find container \"5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0\": container with ID starting with 5ebfe3bc3aadbe94baea41eaed30c4edd0822ac25434168cef97820f080c38b0 not found: ID does not exist"
Jul 07 19:43:38.065775 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.065749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.065927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.065790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.065927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.065829    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.065927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.065856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p97fc\" (UniqueName: \"kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.065927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.065897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.066145 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066029    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home\") pod \"1af783e5-7577-4961-85ae-6170e93ce4ac\" (UID: \"1af783e5-7577-4961-85ae-6170e93ce4ac\") "
Jul 07 19:43:38.066145 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066037    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:43:38.066258 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066226    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:43:38.066372 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066340    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home" (OuterVolumeSpecName: "home") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:43:38.066447 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066382    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.066447 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.066405    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.068199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.068167    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:43:38.068199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.068184    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm" (OuterVolumeSpecName: "dshm") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:43:38.068366 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.068233    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc" (OuterVolumeSpecName: "kube-api-access-p97fc") pod "1af783e5-7577-4961-85ae-6170e93ce4ac" (UID: "1af783e5-7577-4961-85ae-6170e93ce4ac"). InnerVolumeSpecName "kube-api-access-p97fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:43:38.167153 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.167120    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.167153 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.167154    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af783e5-7577-4961-85ae-6170e93ce4ac-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.167327 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.167169    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af783e5-7577-4961-85ae-6170e93ce4ac-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.167327 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.167183    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p97fc\" (UniqueName: \"kubernetes.io/projected/1af783e5-7577-4961-85ae-6170e93ce4ac-kube-api-access-p97fc\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:43:38.344177 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.344142    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:38.347922 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:38.347899    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-reuse-a48272/test-model-a48272-kserve-68d64b7c85-6x92v"]
Jul 07 19:43:39.548378 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:43:39.548347    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1af783e5-7577-4961-85ae-6170e93ce4ac" path="/var/lib/kubelet/pods/1af783e5-7577-4961-85ae-6170e93ce4ac/volumes"
Jul 07 19:44:08.293315 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:08.293273    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:44:09.260915 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:09.260879    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:44:23.313822 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.313774    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" podUID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" containerName="main" containerID="cri-o://a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1" gracePeriod=45
Jul 07 19:44:23.585688 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.585663    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:44:23.776051 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776019    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776066    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776086    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776202    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776221 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm2w9\" (UniqueName: \"kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9\") pod \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\" (UID: \"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c\") "
Jul 07 19:44:23.776465 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776366    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:23.776465 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776380    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home" (OuterVolumeSpecName: "home") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:23.776465 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776453    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:23.776665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776556    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:23.776665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776571    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:23.776665 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.776614    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:23.778437 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.778410    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm" (OuterVolumeSpecName: "dshm") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:23.778541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.778484    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:44:23.778610 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.778537    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9" (OuterVolumeSpecName: "kube-api-access-vm2w9") pod "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" (UID: "9b49ed6b-4b1b-4576-b4be-3b4499f8b57c"). InnerVolumeSpecName "kube-api-access-vm2w9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:44:23.877645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.877544    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:23.877645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.877572    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:23.877645 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:23.877580    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm2w9\" (UniqueName: \"kubernetes.io/projected/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c-kube-api-access-vm2w9\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.246162 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.246130    2579 generic.go:356] "Generic (PLEG): container finished" podID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" containerID="a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1" exitCode=0
Jul 07 19:44:24.246331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.246174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" event={"ID":"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c","Type":"ContainerDied","Data":"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"}
Jul 07 19:44:24.246331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.246175    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"
Jul 07 19:44:24.246331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.246202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9" event={"ID":"9b49ed6b-4b1b-4576-b4be-3b4499f8b57c","Type":"ContainerDied","Data":"c0927b4e86cc5859bf3d1b6a9a43ebb85c7ac8cdda7ab7263b35aeca2d4ffa3b"}
Jul 07 19:44:24.246331 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.246220    2579 scope.go:117] "RemoveContainer" containerID="a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"
Jul 07 19:44:24.255795 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.255777    2579 scope.go:117] "RemoveContainer" containerID="a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"
Jul 07 19:44:24.256083 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:44:24.256061    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1\": container with ID starting with a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1 not found: ID does not exist" containerID="a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"
Jul 07 19:44:24.256144 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.256099    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1"} err="failed to get container status \"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1\": rpc error: code = NotFound desc = could not find container \"a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1\": container with ID starting with a485093043addb350c7d667590cf26841d902b87679553fcd5fdaade1b9600f1 not found: ID does not exist"
Jul 07 19:44:24.268990 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.268964    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:44:24.279894 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.279867    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1aa2d9/test-model-1aa2d9-kserve-6bbbcd7bd5-wgvk9"]
Jul 07 19:44:24.280930 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.280906    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" podUID="77361d36-615d-4e15-9927-c1def34d26df" containerName="main" containerID="cri-o://45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8" gracePeriod=45
Jul 07 19:44:24.530012 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.529990    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:44:24.686050 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686022    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686107    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686145    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfz5d\" (UniqueName: \"kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686175    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs\") pod \"77361d36-615d-4e15-9927-c1def34d26df\" (UID: \"77361d36-615d-4e15-9927-c1def34d26df\") "
Jul 07 19:44:24.686512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686336    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home" (OuterVolumeSpecName: "home") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:24.686512 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686391    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache" (OuterVolumeSpecName: "model-cache") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:24.686634 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686538    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.686634 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686556    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.686752 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.686712    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:24.688679 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.688655    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm" (OuterVolumeSpecName: "dshm") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:44:24.689101 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.689075    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:44:24.689101 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.689081    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d" (OuterVolumeSpecName: "kube-api-access-wfz5d") pod "77361d36-615d-4e15-9927-c1def34d26df" (UID: "77361d36-615d-4e15-9927-c1def34d26df"). InnerVolumeSpecName "kube-api-access-wfz5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:44:24.787611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.787497    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.787611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.787521    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77361d36-615d-4e15-9927-c1def34d26df-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.787611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.787532    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfz5d\" (UniqueName: \"kubernetes.io/projected/77361d36-615d-4e15-9927-c1def34d26df-kube-api-access-wfz5d\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:24.787611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:24.787541    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77361d36-615d-4e15-9927-c1def34d26df-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:44:25.253401 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.253372    2579 generic.go:356] "Generic (PLEG): container finished" podID="77361d36-615d-4e15-9927-c1def34d26df" containerID="45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8" exitCode=0
Jul 07 19:44:25.253572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.253411    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"
Jul 07 19:44:25.253572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.253422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" event={"ID":"77361d36-615d-4e15-9927-c1def34d26df","Type":"ContainerDied","Data":"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"}
Jul 07 19:44:25.253572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.253460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6" event={"ID":"77361d36-615d-4e15-9927-c1def34d26df","Type":"ContainerDied","Data":"1af7b548af7e9f6adbae33562b228227277301e35ae9af6de80388ee20bc2296"}
Jul 07 19:44:25.253572 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.253478    2579 scope.go:117] "RemoveContainer" containerID="45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"
Jul 07 19:44:25.262577 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.262555    2579 scope.go:117] "RemoveContainer" containerID="45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"
Jul 07 19:44:25.262853 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:44:25.262833    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8\": container with ID starting with 45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8 not found: ID does not exist" containerID="45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"
Jul 07 19:44:25.262900 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.262863    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8"} err="failed to get container status \"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8\": rpc error: code = NotFound desc = could not find container \"45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8\": container with ID starting with 45364b74e859bb07a04f60a9400d43671cb77e2126f4bef82d6751524425f9f8 not found: ID does not exist"
Jul 07 19:44:25.275528 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.275507    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:44:25.279909 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.279887    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c47ca8/test-model-c47ca8-kserve-5688579df-47nn6"]
Jul 07 19:44:25.548127 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.548021    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77361d36-615d-4e15-9927-c1def34d26df" path="/var/lib/kubelet/pods/77361d36-615d-4e15-9927-c1def34d26df/volumes"
Jul 07 19:44:25.548569 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:44:25.548430    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" path="/var/lib/kubelet/pods/9b49ed6b-4b1b-4576-b4be-3b4499f8b57c/volumes"
Jul 07 19:46:04.608252 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608220    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608674    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608689    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608704    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77361d36-615d-4e15-9927-c1def34d26df" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608709    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="77361d36-615d-4e15-9927-c1def34d26df" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608720    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af783e5-7577-4961-85ae-6170e93ce4ac" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608725    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af783e5-7577-4961-85ae-6170e93ce4ac" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608801    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="77361d36-615d-4e15-9927-c1def34d26df" containerName="main"
Jul 07 19:46:04.608807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608809    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b49ed6b-4b1b-4576-b4be-3b4499f8b57c" containerName="main"
Jul 07 19:46:04.609061 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.608816    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af783e5-7577-4961-85ae-6170e93ce4ac" containerName="main"
Jul 07 19:46:04.611973 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.611956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.614323 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.614294    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"openshift-service-ca.crt\""
Jul 07 19:46:04.614323 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.614310    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"default-dockercfg-b4g5c\""
Jul 07 19:46:04.614496 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.614317    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"kube-root-ca.crt\""
Jul 07 19:46:04.614496 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.614297    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"auth-test-model-78237e-kserve-self-signed-certs\""
Jul 07 19:46:04.624688 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.624656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:46:04.756780 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.756780 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.756993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxr2m\" (UniqueName: \"kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.756993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.756993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.756993 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.756966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.857961 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.857921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.857971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.857999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxr2m\" (UniqueName: \"kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858156 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858475 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858541 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858497    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.858633 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.858611    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.860667 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.860643    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.860798 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.860705    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.867324 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.867302    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxr2m\" (UniqueName: \"kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m\") pod \"auth-test-model-78237e-kserve-dd454cbc5-sm2r5\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") " pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:04.921659 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:04.921625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:05.063150 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:05.063125    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:46:05.065378 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:46:05.065344    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0f9e71c_5933_4969_b36b_e53d0d10dca8.slice/crio-2443331f51f237e10606d39b893575a1860da3d081ccb5614dd8e73255cf5424 WatchSource:0}: Error finding container 2443331f51f237e10606d39b893575a1860da3d081ccb5614dd8e73255cf5424: Status 404 returned error can't find the container with id 2443331f51f237e10606d39b893575a1860da3d081ccb5614dd8e73255cf5424
Jul 07 19:46:05.708821 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:05.708782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" event={"ID":"d0f9e71c-5933-4969-b36b-e53d0d10dca8","Type":"ContainerStarted","Data":"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"}
Jul 07 19:46:05.709268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:05.708831    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" event={"ID":"d0f9e71c-5933-4969-b36b-e53d0d10dca8","Type":"ContainerStarted","Data":"2443331f51f237e10606d39b893575a1860da3d081ccb5614dd8e73255cf5424"}
Jul 07 19:46:05.728325 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:05.728273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" podStartSLOduration=1.728256421 podStartE2EDuration="1.728256421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:46:05.72764808 +0000 UTC m=+2216.689099358" watchObservedRunningTime="2026-07-07 19:46:05.728256421 +0000 UTC m=+2216.689707697"
Jul 07 19:46:14.922558 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:14.922517    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:14.923100 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:14.922574    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:14.935451 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:14.935419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:15.768432 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:15.768404    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:46:43.215395 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.215320    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:46:43.218718 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.218702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.220821 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.220799    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"openshift-service-ca.crt\""
Jul 07 19:46:43.221108 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.221090    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"kube-root-ca.crt\""
Jul 07 19:46:43.221160 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.221113    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"auth-test-model-0a6be4-kserve-self-signed-certs\""
Jul 07 19:46:43.221319 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.221304    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"default-dockercfg-p9tq9\""
Jul 07 19:46:43.229201 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.229178    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:46:43.321689 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.321810 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.321810 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.321891 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.321891 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn49q\" (UniqueName: \"kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.321975 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.321924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423054 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423181 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423181 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423181 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423352 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn49q\" (UniqueName: \"kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423352 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423560 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423529    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423704 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423570    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.423704 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.423645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.425406 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.425385    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.425849 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.425829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.432077 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.432057    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn49q\" (UniqueName: \"kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q\") pod \"auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") " pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.528227 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.528201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:43.666215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.666190    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:46:43.668413 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:46:43.668389    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c7be57e_43f7_462e_a935_9bd3959fda27.slice/crio-fbbbfb9fba1503438310617d05ab0fe5ae02cbce6ca5be53c8a6797701d7daaa WatchSource:0}: Error finding container fbbbfb9fba1503438310617d05ab0fe5ae02cbce6ca5be53c8a6797701d7daaa: Status 404 returned error can't find the container with id fbbbfb9fba1503438310617d05ab0fe5ae02cbce6ca5be53c8a6797701d7daaa
Jul 07 19:46:43.891002 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.890907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" event={"ID":"9c7be57e-43f7-462e-a935-9bd3959fda27","Type":"ContainerStarted","Data":"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"}
Jul 07 19:46:43.891002 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.890948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" event={"ID":"9c7be57e-43f7-462e-a935-9bd3959fda27","Type":"ContainerStarted","Data":"fbbbfb9fba1503438310617d05ab0fe5ae02cbce6ca5be53c8a6797701d7daaa"}
Jul 07 19:46:43.910297 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:43.910251    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" podStartSLOduration=0.910237507 podStartE2EDuration="910.237507ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:46:43.909158623 +0000 UTC m=+2254.870609902" watchObservedRunningTime="2026-07-07 19:46:43.910237507 +0000 UTC m=+2254.871688783"
Jul 07 19:46:53.529323 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:53.529280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:53.529858 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:53.529345    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:53.549865 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:53.549839    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:46:53.950220 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:46:53.950133    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:48:40.057440 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:40.057407    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:48:41.028463 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:41.028429    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:48:55.078441 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.078393    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" podUID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" containerName="main" containerID="cri-o://5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73" gracePeriod=45
Jul 07 19:48:55.326173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.326149    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:48:55.400118 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400035    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400118 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxr2m\" (UniqueName: \"kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400118 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400106    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400197    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400302    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache\") pod \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\" (UID: \"d0f9e71c-5933-4969-b36b-e53d0d10dca8\") "
Jul 07 19:48:55.400386 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400361    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:55.400574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400531    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:55.400574 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400554    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home" (OuterVolumeSpecName: "home") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:55.400743 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400657    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.400743 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400670    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.400743 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.400689    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.402431 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.402412    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m" (OuterVolumeSpecName: "kube-api-access-dxr2m") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "kube-api-access-dxr2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:48:55.402895 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.402869    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:48:55.402895 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.402887    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm" (OuterVolumeSpecName: "dshm") pod "d0f9e71c-5933-4969-b36b-e53d0d10dca8" (UID: "d0f9e71c-5933-4969-b36b-e53d0d10dca8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:55.501853 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.501818    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0f9e71c-5933-4969-b36b-e53d0d10dca8-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.501853 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.501848    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxr2m\" (UniqueName: \"kubernetes.io/projected/d0f9e71c-5933-4969-b36b-e53d0d10dca8-kube-api-access-dxr2m\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.501853 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.501858    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0f9e71c-5933-4969-b36b-e53d0d10dca8-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:55.506760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.506734    2579 generic.go:356] "Generic (PLEG): container finished" podID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" containerID="5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73" exitCode=0
Jul 07 19:48:55.506892 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.506778    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"
Jul 07 19:48:55.506892 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.506830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" event={"ID":"d0f9e71c-5933-4969-b36b-e53d0d10dca8","Type":"ContainerDied","Data":"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"}
Jul 07 19:48:55.506892 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.506875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5" event={"ID":"d0f9e71c-5933-4969-b36b-e53d0d10dca8","Type":"ContainerDied","Data":"2443331f51f237e10606d39b893575a1860da3d081ccb5614dd8e73255cf5424"}
Jul 07 19:48:55.507053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.506896    2579 scope.go:117] "RemoveContainer" containerID="5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"
Jul 07 19:48:55.515755 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.515718    2579 scope.go:117] "RemoveContainer" containerID="5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"
Jul 07 19:48:55.515970 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:48:55.515952    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73\": container with ID starting with 5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73 not found: ID does not exist" containerID="5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"
Jul 07 19:48:55.516008 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.515979    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73"} err="failed to get container status \"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73\": rpc error: code = NotFound desc = could not find container \"5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73\": container with ID starting with 5452afa6f3ef5424b1f7d61dd74ce098ac0dd1a22f36d3a2aa1a766ff2292d73 not found: ID does not exist"
Jul 07 19:48:55.528995 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.528971    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:48:55.536537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.536513    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/auth-test-model-78237e-kserve-dd454cbc5-sm2r5"]
Jul 07 19:48:55.548453 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:55.548432    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" path="/var/lib/kubelet/pods/d0f9e71c-5933-4969-b36b-e53d0d10dca8/volumes"
Jul 07 19:48:56.048561 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.048522    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" podUID="9c7be57e-43f7-462e-a935-9bd3959fda27" containerName="main" containerID="cri-o://e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7" gracePeriod=45
Jul 07 19:48:56.297388 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.297362    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:48:56.409607 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409517    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409607 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409570    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409793 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409663    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409793 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409718    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn49q\" (UniqueName: \"kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409793 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409955 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409791    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:56.409955 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409798    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir\") pod \"9c7be57e-43f7-462e-a935-9bd3959fda27\" (UID: \"9c7be57e-43f7-462e-a935-9bd3959fda27\") "
Jul 07 19:48:56.409955 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.409921    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home" (OuterVolumeSpecName: "home") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:56.410113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.410058    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:56.410178 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.410157    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.410215 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.410186    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.410308 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.410251    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.411896 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.411870    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm" (OuterVolumeSpecName: "dshm") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:48:56.412016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.411976    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:48:56.412016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.411984    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q" (OuterVolumeSpecName: "kube-api-access-gn49q") pod "9c7be57e-43f7-462e-a935-9bd3959fda27" (UID: "9c7be57e-43f7-462e-a935-9bd3959fda27"). InnerVolumeSpecName "kube-api-access-gn49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:48:56.510856 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.510823    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn49q\" (UniqueName: \"kubernetes.io/projected/9c7be57e-43f7-462e-a935-9bd3959fda27-kube-api-access-gn49q\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.510856 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.510849    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c7be57e-43f7-462e-a935-9bd3959fda27-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.510856 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.510861    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c7be57e-43f7-462e-a935-9bd3959fda27-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:48:56.513407 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.513377    2579 generic.go:356] "Generic (PLEG): container finished" podID="9c7be57e-43f7-462e-a935-9bd3959fda27" containerID="e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7" exitCode=0
Jul 07 19:48:56.513536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.513430    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"
Jul 07 19:48:56.513536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.513451    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" event={"ID":"9c7be57e-43f7-462e-a935-9bd3959fda27","Type":"ContainerDied","Data":"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"}
Jul 07 19:48:56.513536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.513490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn" event={"ID":"9c7be57e-43f7-462e-a935-9bd3959fda27","Type":"ContainerDied","Data":"fbbbfb9fba1503438310617d05ab0fe5ae02cbce6ca5be53c8a6797701d7daaa"}
Jul 07 19:48:56.513536 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.513507    2579 scope.go:117] "RemoveContainer" containerID="e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"
Jul 07 19:48:56.522862 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.522846    2579 scope.go:117] "RemoveContainer" containerID="e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"
Jul 07 19:48:56.523115 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:48:56.523088    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7\": container with ID starting with e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7 not found: ID does not exist" containerID="e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"
Jul 07 19:48:56.523164 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.523116    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7"} err="failed to get container status \"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7\": rpc error: code = NotFound desc = could not find container \"e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7\": container with ID starting with e130aaf99a525409884c9b57d4ad97cef7b37b988e1196d15cfde02ab4edc7f7 not found: ID does not exist"
Jul 07 19:48:56.538312 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.538290    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:48:56.552612 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:56.547874    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/auth-test-model-0a6be4-kserve-f9d5d664c-zc5tn"]
Jul 07 19:48:57.547428 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:48:57.547395    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c7be57e-43f7-462e-a935-9bd3959fda27" path="/var/lib/kubelet/pods/9c7be57e-43f7-462e-a935-9bd3959fda27/volumes"
Jul 07 19:49:00.730713 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.730681    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:49:00.731065 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731023    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c7be57e-43f7-462e-a935-9bd3959fda27" containerName="main"
Jul 07 19:49:00.731065 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731033    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c7be57e-43f7-462e-a935-9bd3959fda27" containerName="main"
Jul 07 19:49:00.731065 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731042    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" containerName="main"
Jul 07 19:49:00.731065 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731047    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" containerName="main"
Jul 07 19:49:00.731192 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731114    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0f9e71c-5933-4969-b36b-e53d0d10dca8" containerName="main"
Jul 07 19:49:00.731192 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.731124    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c7be57e-43f7-462e-a935-9bd3959fda27" containerName="main"
Jul 07 19:49:00.735243 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.735222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.738284 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.738267    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"sub-test-model-78237e-kserve-self-signed-certs\""
Jul 07 19:49:00.738669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.738654    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"kube-root-ca.crt\""
Jul 07 19:49:00.739147 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.739124    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"openshift-service-ca.crt\""
Jul 07 19:49:00.739525 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.739512    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"default-dockercfg-b4g5c\""
Jul 07 19:49:00.750167 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.750142    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:49:00.850951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.850917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.850951 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.850955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.851157 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.851045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.851157 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.851072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.851157 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.851089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.851157 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.851108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm76j\" (UniqueName: \"kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.951991 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.951957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.951991 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.951991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952218 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952218 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952218 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952218 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm76j\" (UniqueName: \"kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952441 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952407    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952516 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952459    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.952567 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.952515    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.954524 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.954498    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.954711 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.954694    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:00.960406 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:00.960386    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm76j\" (UniqueName: \"kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j\") pod \"sub-test-model-78237e-kserve-68c86fb599-jzlkl\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") " pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:01.046220 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:01.046152    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:01.190268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:01.190205    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:49:01.192378 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:49:01.192353    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf943827a_f30e_4825_bddd_d446eb85a853.slice/crio-4adcaf326b57adf1a72c418eb106ea7522750feea077e5f6d5fbe408d84a51ce WatchSource:0}: Error finding container 4adcaf326b57adf1a72c418eb106ea7522750feea077e5f6d5fbe408d84a51ce: Status 404 returned error can't find the container with id 4adcaf326b57adf1a72c418eb106ea7522750feea077e5f6d5fbe408d84a51ce
Jul 07 19:49:01.539637 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:01.539604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" event={"ID":"f943827a-f30e-4825-bddd-d446eb85a853","Type":"ContainerStarted","Data":"0d2c767d1d68c3953bde5e1960f23faf24ccf9ad581ff1a871961afe7b70a4aa"}
Jul 07 19:49:01.539637 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:01.539644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" event={"ID":"f943827a-f30e-4825-bddd-d446eb85a853","Type":"ContainerStarted","Data":"4adcaf326b57adf1a72c418eb106ea7522750feea077e5f6d5fbe408d84a51ce"}
Jul 07 19:49:01.563039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:01.562990    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" podStartSLOduration=1.562976172 podStartE2EDuration="1.562976172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:49:01.561514642 +0000 UTC m=+2392.522965920" watchObservedRunningTime="2026-07-07 19:49:01.562976172 +0000 UTC m=+2392.524427448"
Jul 07 19:49:11.046672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:11.046638    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:11.047112 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:11.046684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:11.059280 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:11.059256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:11.597234 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:11.597200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:49:23.519807 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.519769    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:49:23.523316 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.523298    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.525828 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.525806    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"kube-root-ca.crt\""
Jul 07 19:49:23.525937 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.525850    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"openshift-service-ca.crt\""
Jul 07 19:49:23.525937 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.525804    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"default-dockercfg-p9tq9\""
Jul 07 19:49:23.525937 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.525813    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"sub-test-model-0a6be4-kserve-self-signed-certs\""
Jul 07 19:49:23.537385 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.537351    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:49:23.552248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.552466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.552466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.552466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.552466 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.552635 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.552481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dnwf\" (UniqueName: \"kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653165 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dnwf\" (UniqueName: \"kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653399 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653775 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653775 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653731    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.653874 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.653846    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.656249 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.656221    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.656379 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.656350    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.665146 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.665120    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dnwf\" (UniqueName: \"kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf\") pod \"sub-test-model-0a6be4-kserve-7985b49995-xqfpl\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") " pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.834611 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.834499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:23.971113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:23.971089    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:49:23.973295 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:49:23.973269    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c9a3b43_60f9_41dc_989c_2a8a8e675430.slice/crio-981729d9e792d4b7451b7cb4efb87a2c4f30bfbf5d854dd82f4b467302c83356 WatchSource:0}: Error finding container 981729d9e792d4b7451b7cb4efb87a2c4f30bfbf5d854dd82f4b467302c83356: Status 404 returned error can't find the container with id 981729d9e792d4b7451b7cb4efb87a2c4f30bfbf5d854dd82f4b467302c83356
Jul 07 19:49:24.647770 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:24.647738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" event={"ID":"5c9a3b43-60f9-41dc-989c-2a8a8e675430","Type":"ContainerStarted","Data":"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"}
Jul 07 19:49:24.648199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:24.647781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" event={"ID":"5c9a3b43-60f9-41dc-989c-2a8a8e675430","Type":"ContainerStarted","Data":"981729d9e792d4b7451b7cb4efb87a2c4f30bfbf5d854dd82f4b467302c83356"}
Jul 07 19:49:24.666762 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:24.666720    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" podStartSLOduration=1.666704712 podStartE2EDuration="1.666704712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:49:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:49:24.665949075 +0000 UTC m=+2415.627400353" watchObservedRunningTime="2026-07-07 19:49:24.666704712 +0000 UTC m=+2415.628156043"
Jul 07 19:49:33.835229 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:33.835145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:33.835229 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:33.835199    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:33.847767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:33.847740    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:49:34.707095 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:49:34.707064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:50:20.727784 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:20.727746    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:50:21.659203 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:21.659170    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:50:29.908779 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:29.908745    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:50:29.911966 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:29.911948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:29.913810 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:29.913784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-78237e\"/\"rate-test-model-78237e-kserve-self-signed-certs\""
Jul 07 19:50:29.923238 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:29.923214    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:50:30.056025 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.055993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.056196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.056045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.056196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.056078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.056196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.056112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.056196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.056148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.056196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.056184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw25q\" (UniqueName: \"kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157404 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw25q\" (UniqueName: \"kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157669 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157869 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157847    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.157975 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157954    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.158036 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.157996    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.159999 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.159939    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.160199 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.160183    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.170571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.170542    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw25q\" (UniqueName: \"kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q\") pod \"rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") " pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.220533 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.220497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:30.362861 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.362834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:50:30.364523 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:50:30.364498    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09377798_2c2b_41a5_ad28_e49b4f5a40a3.slice/crio-8538491c7d4b8f8365647f61ca3695da7e5edac380fb209e8dbc6c0499118695 WatchSource:0}: Error finding container 8538491c7d4b8f8365647f61ca3695da7e5edac380fb209e8dbc6c0499118695: Status 404 returned error can't find the container with id 8538491c7d4b8f8365647f61ca3695da7e5edac380fb209e8dbc6c0499118695
Jul 07 19:50:30.960310 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.960275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" event={"ID":"09377798-2c2b-41a5-ad28-e49b4f5a40a3","Type":"ContainerStarted","Data":"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"}
Jul 07 19:50:30.960310 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.960317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" event={"ID":"09377798-2c2b-41a5-ad28-e49b4f5a40a3","Type":"ContainerStarted","Data":"8538491c7d4b8f8365647f61ca3695da7e5edac380fb209e8dbc6c0499118695"}
Jul 07 19:50:30.986244 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:30.986194    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" podStartSLOduration=1.986174884 podStartE2EDuration="1.986174884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:50:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:50:30.984343535 +0000 UTC m=+2481.945794815" watchObservedRunningTime="2026-07-07 19:50:30.986174884 +0000 UTC m=+2481.947626161"
Jul 07 19:50:35.747929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:35.747877    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" podUID="f943827a-f30e-4825-bddd-d446eb85a853" containerName="main" containerID="cri-o://0d2c767d1d68c3953bde5e1960f23faf24ccf9ad581ff1a871961afe7b70a4aa" gracePeriod=45
Jul 07 19:50:35.986327 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:35.986128    2579 generic.go:356] "Generic (PLEG): container finished" podID="f943827a-f30e-4825-bddd-d446eb85a853" containerID="0d2c767d1d68c3953bde5e1960f23faf24ccf9ad581ff1a871961afe7b70a4aa" exitCode=0
Jul 07 19:50:35.986327 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:35.986198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" event={"ID":"f943827a-f30e-4825-bddd-d446eb85a853","Type":"ContainerDied","Data":"0d2c767d1d68c3953bde5e1960f23faf24ccf9ad581ff1a871961afe7b70a4aa"}
Jul 07 19:50:36.007187 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.007126    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:50:36.113961 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.113926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.113995    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114094    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm76j\" (UniqueName: \"kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114117    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114133    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114173 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114161    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm\") pod \"f943827a-f30e-4825-bddd-d446eb85a853\" (UID: \"f943827a-f30e-4825-bddd-d446eb85a853\") "
Jul 07 19:50:36.114434 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114275    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home" (OuterVolumeSpecName: "home") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:36.114434 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114389    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache" (OuterVolumeSpecName: "model-cache") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:36.114434 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114402    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:36.114554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114542    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.114646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114560    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.114646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.114573    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.116292 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.116269    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm" (OuterVolumeSpecName: "dshm") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:36.116401 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.116300    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:50:36.116401 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.116371    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j" (OuterVolumeSpecName: "kube-api-access-qm76j") pod "f943827a-f30e-4825-bddd-d446eb85a853" (UID: "f943827a-f30e-4825-bddd-d446eb85a853"). InnerVolumeSpecName "kube-api-access-qm76j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:50:36.215208 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.215167    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f943827a-f30e-4825-bddd-d446eb85a853-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.215208 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.215200    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f943827a-f30e-4825-bddd-d446eb85a853-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.215208 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.215216    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm76j\" (UniqueName: \"kubernetes.io/projected/f943827a-f30e-4825-bddd-d446eb85a853-kube-api-access-qm76j\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:36.679637 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.679577    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" podUID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" containerName="main" containerID="cri-o://89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d" gracePeriod=45
Jul 07 19:50:36.940141 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.940076    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:50:36.993322 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.993280    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"
Jul 07 19:50:36.993322 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.993306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl" event={"ID":"f943827a-f30e-4825-bddd-d446eb85a853","Type":"ContainerDied","Data":"4adcaf326b57adf1a72c418eb106ea7522750feea077e5f6d5fbe408d84a51ce"}
Jul 07 19:50:36.993631 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.993384    2579 scope.go:117] "RemoveContainer" containerID="0d2c767d1d68c3953bde5e1960f23faf24ccf9ad581ff1a871961afe7b70a4aa"
Jul 07 19:50:36.994877 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.994854    2579 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" containerID="89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d" exitCode=0
Jul 07 19:50:36.994987 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.994894    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"
Jul 07 19:50:36.994987 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.994897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" event={"ID":"5c9a3b43-60f9-41dc-989c-2a8a8e675430","Type":"ContainerDied","Data":"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"}
Jul 07 19:50:36.995084 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:36.995006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl" event={"ID":"5c9a3b43-60f9-41dc-989c-2a8a8e675430","Type":"ContainerDied","Data":"981729d9e792d4b7451b7cb4efb87a2c4f30bfbf5d854dd82f4b467302c83356"}
Jul 07 19:50:37.001968 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.001947    2579 scope.go:117] "RemoveContainer" containerID="89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"
Jul 07 19:50:37.010242 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.010223    2579 scope.go:117] "RemoveContainer" containerID="89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"
Jul 07 19:50:37.010753 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:50:37.010721    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d\": container with ID starting with 89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d not found: ID does not exist" containerID="89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"
Jul 07 19:50:37.010858 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.010761    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d"} err="failed to get container status \"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d\": rpc error: code = NotFound desc = could not find container \"89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d\": container with ID starting with 89e6a34c7d09e3de9f7ba855c12dad606ecf71a3a8764647fb4149e951ab3b6d not found: ID does not exist"
Jul 07 19:50:37.021899 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.021876    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:50:37.022953 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.022934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023012 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023049 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023021    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023049 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023040    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023148 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023100    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dnwf\" (UniqueName: \"kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023148 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir\") pod \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\" (UID: \"5c9a3b43-60f9-41dc-989c-2a8a8e675430\") "
Jul 07 19:50:37.023245 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023159    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home" (OuterVolumeSpecName: "home") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:37.023374 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023354    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:37.023374 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023362    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:37.023549 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023531    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.023629 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023552    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.023629 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.023561    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.025336 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.025292    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm" (OuterVolumeSpecName: "dshm") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:50:37.025894 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.025854    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf" (OuterVolumeSpecName: "kube-api-access-6dnwf") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "kube-api-access-6dnwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:50:37.026005 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.025961    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c9a3b43-60f9-41dc-989c-2a8a8e675430" (UID: "5c9a3b43-60f9-41dc-989c-2a8a8e675430"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:50:37.027795 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.027778    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/sub-test-model-78237e-kserve-68c86fb599-jzlkl"]
Jul 07 19:50:37.124600 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.124555    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3b43-60f9-41dc-989c-2a8a8e675430-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.124600 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.124584    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3b43-60f9-41dc-989c-2a8a8e675430-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.124801 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.124616    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dnwf\" (UniqueName: \"kubernetes.io/projected/5c9a3b43-60f9-41dc-989c-2a8a8e675430-kube-api-access-6dnwf\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:50:37.318272 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.318238    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:50:37.323119 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.323090    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/sub-test-model-0a6be4-kserve-7985b49995-xqfpl"]
Jul 07 19:50:37.548859 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.548828    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" path="/var/lib/kubelet/pods/5c9a3b43-60f9-41dc-989c-2a8a8e675430/volumes"
Jul 07 19:50:37.549222 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:37.549209    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f943827a-f30e-4825-bddd-d446eb85a853" path="/var/lib/kubelet/pods/f943827a-f30e-4825-bddd-d446eb85a853/volumes"
Jul 07 19:50:40.220815 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:40.220778    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:40.221196 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:40.220827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:40.232760 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:40.232737    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:50:41.027009 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:50:41.026976    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:51:01.292823 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.292744    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:51:01.293298 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293178    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" containerName="main"
Jul 07 19:51:01.293298 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293196    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" containerName="main"
Jul 07 19:51:01.293298 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293230    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f943827a-f30e-4825-bddd-d446eb85a853" containerName="main"
Jul 07 19:51:01.293298 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293239    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f943827a-f30e-4825-bddd-d446eb85a853" containerName="main"
Jul 07 19:51:01.293521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293333    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3b43-60f9-41dc-989c-2a8a8e675430" containerName="main"
Jul 07 19:51:01.293521 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.293352    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f943827a-f30e-4825-bddd-d446eb85a853" containerName="main"
Jul 07 19:51:01.296606 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.296571    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.298863 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.298839    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"openshift-service-ca.crt\""
Jul 07 19:51:01.298974 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.298841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"kube-root-ca.crt\""
Jul 07 19:51:01.299268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.299248    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"default-dockercfg-p9tq9\""
Jul 07 19:51:01.299268 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.299262    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a6be4\"/\"rate-test-model-0a6be4-kserve-self-signed-certs\""
Jul 07 19:51:01.307514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.307485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:51:01.340006 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.339978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.340006 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.340010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.340217 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.340035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.340217 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.340162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.340217 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.340195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.340317 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.340245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4zc4\" (UniqueName: \"kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.440918 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.440877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.440932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4zc4\" (UniqueName: \"kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.440968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.440985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.441007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.441072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441486 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.441462    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441550 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.441481    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.441619 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.441545    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.443490 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.443464    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.443742 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.443723    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.451164 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.451129    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4zc4\" (UniqueName: \"kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4\") pod \"rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") " pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.605913 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.605827    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:01.754671 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:01.754646    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:51:01.756798 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:51:01.756768    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd379b26_d352_48cb_bd18_ed868b1a5c50.slice/crio-bb46c70698f20763bc0e321256a8958988f95f12236b2968037e4f9e22c5b4d9 WatchSource:0}: Error finding container bb46c70698f20763bc0e321256a8958988f95f12236b2968037e4f9e22c5b4d9: Status 404 returned error can't find the container with id bb46c70698f20763bc0e321256a8958988f95f12236b2968037e4f9e22c5b4d9
Jul 07 19:51:02.118246 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:02.118204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" event={"ID":"bd379b26-d352-48cb-bd18-ed868b1a5c50","Type":"ContainerStarted","Data":"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"}
Jul 07 19:51:02.118246 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:02.118244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" event={"ID":"bd379b26-d352-48cb-bd18-ed868b1a5c50","Type":"ContainerStarted","Data":"bb46c70698f20763bc0e321256a8958988f95f12236b2968037e4f9e22c5b4d9"}
Jul 07 19:51:02.149166 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:02.149120    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" podStartSLOduration=1.149105635 podStartE2EDuration="1.149105635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:51:02.147291403 +0000 UTC m=+2513.108742685" watchObservedRunningTime="2026-07-07 19:51:02.149105635 +0000 UTC m=+2513.110556911"
Jul 07 19:51:11.606145 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:11.606109    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:11.606550 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:11.606158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:11.618434 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:11.618408    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:12.176799 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:12.176772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:51:50.614159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:50.614126    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:51:51.491927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:51:51.491889    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:52:05.634121 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:05.634083    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" podUID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" containerName="main" containerID="cri-o://4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062" gracePeriod=45
Jul 07 19:52:05.885786 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:05.885729    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:52:06.017445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017445 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017451    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017472    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017525    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017568    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017716 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw25q\" (UniqueName: \"kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q\") pod \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\" (UID: \"09377798-2c2b-41a5-ad28-e49b4f5a40a3\") "
Jul 07 19:52:06.017927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017706    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.017927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017757    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home" (OuterVolumeSpecName: "home") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.017927 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017864    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.018039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.017999    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.018039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.018017    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.018039 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.018030    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.019767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.019734    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:52:06.019767 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.019753    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q" (OuterVolumeSpecName: "kube-api-access-rw25q") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "kube-api-access-rw25q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:52:06.019914 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.019795    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm" (OuterVolumeSpecName: "dshm") pod "09377798-2c2b-41a5-ad28-e49b4f5a40a3" (UID: "09377798-2c2b-41a5-ad28-e49b4f5a40a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.119000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.118968    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09377798-2c2b-41a5-ad28-e49b4f5a40a3-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.119000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.118996    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw25q\" (UniqueName: \"kubernetes.io/projected/09377798-2c2b-41a5-ad28-e49b4f5a40a3-kube-api-access-rw25q\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.119000 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.119006    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09377798-2c2b-41a5-ad28-e49b4f5a40a3-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.421991 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.421964    2579 generic.go:356] "Generic (PLEG): container finished" podID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" containerID="4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062" exitCode=0
Jul 07 19:52:06.422159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.422007    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"
Jul 07 19:52:06.422159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.422048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" event={"ID":"09377798-2c2b-41a5-ad28-e49b4f5a40a3","Type":"ContainerDied","Data":"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"}
Jul 07 19:52:06.422159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.422092    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs" event={"ID":"09377798-2c2b-41a5-ad28-e49b4f5a40a3","Type":"ContainerDied","Data":"8538491c7d4b8f8365647f61ca3695da7e5edac380fb209e8dbc6c0499118695"}
Jul 07 19:52:06.422159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.422109    2579 scope.go:117] "RemoveContainer" containerID="4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"
Jul 07 19:52:06.431661 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.431642    2579 scope.go:117] "RemoveContainer" containerID="4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"
Jul 07 19:52:06.431938 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:52:06.431921    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062\": container with ID starting with 4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062 not found: ID does not exist" containerID="4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"
Jul 07 19:52:06.431994 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.431945    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062"} err="failed to get container status \"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062\": rpc error: code = NotFound desc = could not find container \"4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062\": container with ID starting with 4a7d5fddc5cda527f1c0a1ce0abc7759b2a1586a39b98ca6c452535357514062 not found: ID does not exist"
Jul 07 19:52:06.443871 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.443850    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:52:06.449217 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.449195    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-78237e/rate-test-model-78237e-kserve-c5dc8b8c9-kgwbs"]
Jul 07 19:52:06.511672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.511645    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" podUID="bd379b26-d352-48cb-bd18-ed868b1a5c50" containerName="main" containerID="cri-o://0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36" gracePeriod=45
Jul 07 19:52:06.765986 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.765964    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:52:06.825094 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825068    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825237 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825114    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825237 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825237 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825224    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825257    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825403 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825313    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4zc4\" (UniqueName: \"kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4\") pod \"bd379b26-d352-48cb-bd18-ed868b1a5c50\" (UID: \"bd379b26-d352-48cb-bd18-ed868b1a5c50\") "
Jul 07 19:52:06.825487 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825402    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.825537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825503    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home" (OuterVolumeSpecName: "home") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.825610 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825530    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.825682 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825647    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-model-cache\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.825682 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825662    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-home\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.825682 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.825670    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-tmp-dir\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.827265 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.827246    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm" (OuterVolumeSpecName: "dshm") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 19:52:06.827362 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.827342    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 19:52:06.827441 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.827424    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4" (OuterVolumeSpecName: "kube-api-access-l4zc4") pod "bd379b26-d352-48cb-bd18-ed868b1a5c50" (UID: "bd379b26-d352-48cb-bd18-ed868b1a5c50"). InnerVolumeSpecName "kube-api-access-l4zc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 19:52:06.926942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.926909    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd379b26-d352-48cb-bd18-ed868b1a5c50-dshm\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.926942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.926935    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd379b26-d352-48cb-bd18-ed868b1a5c50-tls-certs\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:06.926942 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:06.926945    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4zc4\" (UniqueName: \"kubernetes.io/projected/bd379b26-d352-48cb-bd18-ed868b1a5c50-kube-api-access-l4zc4\") on node \"ip-10-0-136-217.ec2.internal\" DevicePath \"\""
Jul 07 19:52:07.428473 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.428441    2579 generic.go:356] "Generic (PLEG): container finished" podID="bd379b26-d352-48cb-bd18-ed868b1a5c50" containerID="0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36" exitCode=0
Jul 07 19:52:07.428662 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.428480    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"
Jul 07 19:52:07.428662 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.428536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" event={"ID":"bd379b26-d352-48cb-bd18-ed868b1a5c50","Type":"ContainerDied","Data":"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"}
Jul 07 19:52:07.428662 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.428576    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c" event={"ID":"bd379b26-d352-48cb-bd18-ed868b1a5c50","Type":"ContainerDied","Data":"bb46c70698f20763bc0e321256a8958988f95f12236b2968037e4f9e22c5b4d9"}
Jul 07 19:52:07.428662 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.428610    2579 scope.go:117] "RemoveContainer" containerID="0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"
Jul 07 19:52:07.437108 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.437088    2579 scope.go:117] "RemoveContainer" containerID="0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"
Jul 07 19:52:07.437342 ip-10-0-136-217 kubenswrapper[2579]: E0707 19:52:07.437325    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36\": container with ID starting with 0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36 not found: ID does not exist" containerID="0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"
Jul 07 19:52:07.437396 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.437350    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36"} err="failed to get container status \"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36\": rpc error: code = NotFound desc = could not find container \"0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36\": container with ID starting with 0b0480564b9f9fc15d5361a21910c951a6a60a5d64e6ba0a811d422486703c36 not found: ID does not exist"
Jul 07 19:52:07.450472 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.450446    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:52:07.452937 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.452915    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0a6be4/rate-test-model-0a6be4-kserve-6bb59d88b7-5fz4c"]
Jul 07 19:52:07.548285 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.548254    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" path="/var/lib/kubelet/pods/09377798-2c2b-41a5-ad28-e49b4f5a40a3/volumes"
Jul 07 19:52:07.548672 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:07.548658    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd379b26-d352-48cb-bd18-ed868b1a5c50" path="/var/lib/kubelet/pods/bd379b26-d352-48cb-bd18-ed868b1a5c50/volumes"
Jul 07 19:52:25.129894 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:25.129864    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75847b84df-w4fd4_1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64/copy-manifests/0.log"
Jul 07 19:52:25.136802 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:25.136779    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75847b84df-w4fd4_1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64/manager/0.log"
Jul 07 19:52:26.853429 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:26.853400    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gxrzx_3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7/kuadrant-console-plugin/0.log"
Jul 07 19:52:27.090996 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:27.090968    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mttmz_d2d1bd53-29ba-4ac0-86e0-afd1b1e04766/manager/0.log"
Jul 07 19:52:28.027514 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:28.027484    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598d7cb498-b2rmr_6230ef1f-26c1-4e0d-8988-9da05bb35cc0/kube-auth-proxy/0.log"
Jul 07 19:52:28.377123 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:28.377044    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-txnbg_20f6a441-f584-4a2e-b1da-6b022d8c1a5d/payload-processing/0.log"
Jul 07 19:52:29.047975 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:29.047938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2bmfr_a4910f70-d42d-4e6e-a61b-038c58d9f77c/main/0.log"
Jul 07 19:52:29.269873 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:29.269842    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xzf4f_d02ead4d-41fb-496e-a5bd-af8c183a5ceb/main/0.log"
Jul 07 19:52:29.380471 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:29.380402    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-rb7q2_e1be8192-9e2b-41c4-a7a2-5c434ef247d8/main/0.log"
Jul 07 19:52:36.631248 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:36.631219    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wp8nm_fb0b3d25-d5e0-42bd-9eff-9909e26175d5/global-pull-secret-syncer/0.log"
Jul 07 19:52:36.760309 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:36.760277    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v9nhr_7be1de55-29fc-40d0-a4a7-21a2b9a95c6a/konnectivity-agent/0.log"
Jul 07 19:52:36.788651 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:36.788624    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-217.ec2.internal_e0242a47c8f42188a150225d6e13e270/haproxy/0.log"
Jul 07 19:52:42.452116 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:42.452011    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gxrzx_3b6b2ca7-8b88-4299-a2b1-86cbd2d954c7/kuadrant-console-plugin/0.log"
Jul 07 19:52:42.667623 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:42.667574    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mttmz_d2d1bd53-29ba-4ac0-86e0-afd1b1e04766/manager/0.log"
Jul 07 19:52:44.823355 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:44.823330    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hw4cb_c5bceb52-bdd2-413e-8871-dcefa851d0d9/cluster-monitoring-operator/0.log"
Jul 07 19:52:44.849657 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:44.849627    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bfx9p_ef5eea5f-b32a-4df8-8f0f-ea91359c93af/kube-state-metrics/0.log"
Jul 07 19:52:44.875776 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:44.875746    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bfx9p_ef5eea5f-b32a-4df8-8f0f-ea91359c93af/kube-rbac-proxy-main/0.log"
Jul 07 19:52:44.902267 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:44.902246    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bfx9p_ef5eea5f-b32a-4df8-8f0f-ea91359c93af/kube-rbac-proxy-self/0.log"
Jul 07 19:52:45.014823 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.014786    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jw24g_5ea69466-2fa0-43fb-8747-992942f1bc2b/node-exporter/0.log"
Jul 07 19:52:45.040379 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.040356    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jw24g_5ea69466-2fa0-43fb-8747-992942f1bc2b/kube-rbac-proxy/0.log"
Jul 07 19:52:45.082562 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.082543    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jw24g_5ea69466-2fa0-43fb-8747-992942f1bc2b/init-textfile/0.log"
Jul 07 19:52:45.310717 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.310581    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-b9lxh_2563567e-7226-4607-b9af-49b5d460b9eb/kube-rbac-proxy-main/0.log"
Jul 07 19:52:45.339680 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.339658    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-b9lxh_2563567e-7226-4607-b9af-49b5d460b9eb/kube-rbac-proxy-self/0.log"
Jul 07 19:52:45.367564 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.367542    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-b9lxh_2563567e-7226-4607-b9af-49b5d460b9eb/openshift-state-metrics/0.log"
Jul 07 19:52:45.414285 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.414265    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/prometheus/0.log"
Jul 07 19:52:45.439048 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.439019    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/config-reloader/0.log"
Jul 07 19:52:45.471618 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.471580    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/thanos-sidecar/0.log"
Jul 07 19:52:45.496571 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.496553    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/kube-rbac-proxy-web/0.log"
Jul 07 19:52:45.525537 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.525514    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/kube-rbac-proxy/0.log"
Jul 07 19:52:45.553159 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.553132    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/kube-rbac-proxy-thanos/0.log"
Jul 07 19:52:45.580420 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.580370    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc7fcbc7-80cd-40e7-8061-4084348b43d4/init-config-reloader/0.log"
Jul 07 19:52:45.676016 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.675988    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jrn4n_d89b74ce-acc0-4fe2-867d-827281431cb6/prometheus-operator-admission-webhook/0.log"
Jul 07 19:52:45.824335 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.824306    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/thanos-query/0.log"
Jul 07 19:52:45.848094 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.848034    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/kube-rbac-proxy-web/0.log"
Jul 07 19:52:45.884710 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.884689    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/kube-rbac-proxy/0.log"
Jul 07 19:52:45.910646 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.910616    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/prom-label-proxy/0.log"
Jul 07 19:52:45.935720 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.935701    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/kube-rbac-proxy-rules/0.log"
Jul 07 19:52:45.964100 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:45.964061    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c7f79947-sdn5w_b60535f9-a6c9-4414-9a5a-27316498bfae/kube-rbac-proxy-metrics/0.log"
Jul 07 19:52:46.705133 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705099    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"]
Jul 07 19:52:46.705519 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705505    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd379b26-d352-48cb-bd18-ed868b1a5c50" containerName="main"
Jul 07 19:52:46.705519 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705520    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd379b26-d352-48cb-bd18-ed868b1a5c50" containerName="main"
Jul 07 19:52:46.705631 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705535    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" containerName="main"
Jul 07 19:52:46.705631 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" containerName="main"
Jul 07 19:52:46.705715 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705652    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd379b26-d352-48cb-bd18-ed868b1a5c50" containerName="main"
Jul 07 19:52:46.705715 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.705667    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="09377798-2c2b-41a5-ad28-e49b4f5a40a3" containerName="main"
Jul 07 19:52:46.708940 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.708914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.710816 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.710791    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j2nl9\"/\"default-dockercfg-qqkct\""
Jul 07 19:52:46.710919 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.710828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j2nl9\"/\"openshift-service-ca.crt\""
Jul 07 19:52:46.710919 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.710882    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j2nl9\"/\"kube-root-ca.crt\""
Jul 07 19:52:46.718463 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.718440    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"]
Jul 07 19:52:46.780469 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.780444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kj5v\" (UniqueName: \"kubernetes.io/projected/04e31fd6-7798-4278-8dc2-952d3f0dce6d-kube-api-access-8kj5v\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.780621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.780477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-sys\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.780621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.780511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-podres\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.780621 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.780610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-proc\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.780740 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.780639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-lib-modules\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881554 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-podres\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-proc\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-lib-modules\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kj5v\" (UniqueName: \"kubernetes.io/projected/04e31fd6-7798-4278-8dc2-952d3f0dce6d-kube-api-access-8kj5v\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881669    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-proc\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-sys\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881696    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-podres\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881768    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-lib-modules\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.881929 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.881775    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04e31fd6-7798-4278-8dc2-952d3f0dce6d-sys\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:46.890310 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:46.890289    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kj5v\" (UniqueName: \"kubernetes.io/projected/04e31fd6-7798-4278-8dc2-952d3f0dce6d-kube-api-access-8kj5v\") pod \"perf-node-gather-daemonset-n2j4m\" (UID: \"04e31fd6-7798-4278-8dc2-952d3f0dce6d\") " pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:47.019686 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.019656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:47.163113 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.163089    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"]
Jul 07 19:52:47.164762 ip-10-0-136-217 kubenswrapper[2579]: W0707 19:52:47.164736    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod04e31fd6_7798_4278_8dc2_952d3f0dce6d.slice/crio-1eeff4f05d668815366b7f0dc53417509a755293ecf610218eb5afd09af1e471 WatchSource:0}: Error finding container 1eeff4f05d668815366b7f0dc53417509a755293ecf610218eb5afd09af1e471: Status 404 returned error can't find the container with id 1eeff4f05d668815366b7f0dc53417509a755293ecf610218eb5afd09af1e471
Jul 07 19:52:47.615703 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.615663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m" event={"ID":"04e31fd6-7798-4278-8dc2-952d3f0dce6d","Type":"ContainerStarted","Data":"1ce34728c702cf4c91852bf3a7e77e35e942c6413390e5d30072881df4b36824"}
Jul 07 19:52:47.615882 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.615718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m" event={"ID":"04e31fd6-7798-4278-8dc2-952d3f0dce6d","Type":"ContainerStarted","Data":"1eeff4f05d668815366b7f0dc53417509a755293ecf610218eb5afd09af1e471"}
Jul 07 19:52:47.615882 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.615755    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:47.635308 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:47.635264    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m" podStartSLOduration=1.635250227 podStartE2EDuration="1.635250227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:52:47.633751071 +0000 UTC m=+2618.595202342" watchObservedRunningTime="2026-07-07 19:52:47.635250227 +0000 UTC m=+2618.596701537"
Jul 07 19:52:48.247628 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:48.247579    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7dcb5584c5-4zngf_04891dbd-6f31-4bd5-942d-9da5ec22616a/console/0.log"
Jul 07 19:52:48.284245 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:48.284217    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-xc9kj_11398cfd-4692-40d0-bf1b-f84414c94184/download-server/0.log"
Jul 07 19:52:50.186050 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:50.186026    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-whk7g_ae84ed10-ccc5-4a89-9374-bbbcda75246a/dns/0.log"
Jul 07 19:52:50.213334 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:50.213313    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-whk7g_ae84ed10-ccc5-4a89-9374-bbbcda75246a/kube-rbac-proxy/0.log"
Jul 07 19:52:50.299012 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:50.298945    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l69s9_5efe5858-ff97-4429-b89f-85e7cc3cf1a4/dns-node-resolver/0.log"
Jul 07 19:52:50.910197 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:50.910174    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-85qcf_ed443871-0f3a-487a-afe6-38fe7fc9ae81/node-ca/0.log"
Jul 07 19:52:52.278686 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:52.278656    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598d7cb498-b2rmr_6230ef1f-26c1-4e0d-8988-9da05bb35cc0/kube-auth-proxy/0.log"
Jul 07 19:52:52.434814 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:52.434789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-txnbg_20f6a441-f584-4a2e-b1da-6b022d8c1a5d/payload-processing/0.log"
Jul 07 19:52:53.213655 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:53.213630    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g58w5_720a723a-49e6-4c43-9b03-dfef193e6930/serve-healthcheck-canary/0.log"
Jul 07 19:52:53.630098 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:53.630074    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j2nl9/perf-node-gather-daemonset-n2j4m"
Jul 07 19:52:53.884919 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:53.884846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5jz4_a4156ab8-26bf-4357-bab3-289a19fd4bb4/kube-rbac-proxy/0.log"
Jul 07 19:52:53.916053 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:53.916033    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5jz4_a4156ab8-26bf-4357-bab3-289a19fd4bb4/exporter/0.log"
Jul 07 19:52:53.942871 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:53.942839    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5jz4_a4156ab8-26bf-4357-bab3-289a19fd4bb4/extractor/0.log"
Jul 07 19:52:57.412249 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:57.412219    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75847b84df-w4fd4_1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64/manager/0.log"
Jul 07 19:52:57.437525 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:57.437499    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75847b84df-w4fd4_1a7d35c7-bcbf-4c1d-90aa-f54e9c35be64/copy-manifests/0.log"
Jul 07 19:52:57.465479 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:57.465455    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598d7cb498-b2rmr_6230ef1f-26c1-4e0d-8988-9da05bb35cc0/kube-auth-proxy/0.log"
Jul 07 19:52:57.601394 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:57.601362    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-txnbg_20f6a441-f584-4a2e-b1da-6b022d8c1a5d/payload-processing/0.log"
Jul 07 19:52:59.009979 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:59.009952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76668d465c-g7jbt_cb6dff71-aa05-4b75-9511-9d86c82b05a7/manager/0.log"
Jul 07 19:52:59.160338 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:59.160312    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598d7cb498-b2rmr_6230ef1f-26c1-4e0d-8988-9da05bb35cc0/kube-auth-proxy/0.log"
Jul 07 19:52:59.289071 ip-10-0-136-217 kubenswrapper[2579]: I0707 19:52:59.289006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-txnbg_20f6a441-f584-4a2e-b1da-6b022d8c1a5d/payload-processing/0.log"